robots

robots

Modules:

Name	Description
abstract
bimanual_yam	Bimanual YAM robot implementation for the framework.
floating_robotiq
floating_rum
franka
i2rt_yam	i2rt YAM robot implementation for the framework.
rby1
robot_views

abstract

Classes:

Name	Description
Robot

Attributes:

Name	Type	Description
log

log module-attribute

log = getLogger(__name__)

Robot

Robot(mj_data: MjData, exp_config: MlSpacesExpConfig)

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the robot definistion and current simulation state	required

Methods:

Name	Description
action_dim	sum of the commanded joints, which defines the action space of the robot.
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified location (x-y-yaw) in the world.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers	dict[str, Controller]	One or more controllers for the robot joints / wheels / etc.
exp_config
kinematics	MlSpacesKinematics	kinematic solver for the robot
mj_data
mj_model
namespace	str	robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics	ParallelKinematics	parallel kinematic solver for the robot
robot_view	RobotView	robot view for interfacing with joints / links / actuators
state_dim	int	sum of all the joints of interest, which defines the state of the robot.

Source code in molmo_spaces/robots/abstract.py

def __init__(self, mj_data: MjData, exp_config: "MlSpacesExpConfig"):
    """
    Args:
        mj_data: The MuJoCo data structure containing the robot definistion and current simulation state
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data
    self.exp_config = exp_config
    self._last_unnoised_cmd_joint_pos: dict[str, np.ndarray] | None = None

controllers abstractmethod property

controllers: dict[str, Controller]

One or more controllers for the robot joints / wheels / etc.

exp_config instance-attribute

exp_config = exp_config

kinematics abstractmethod property

kinematics: MlSpacesKinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

namespace abstractmethod property

namespace: str

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics abstractmethod property

parallel_kinematics: ParallelKinematics

parallel kinematic solver for the robot

robot_view abstractmethod property

robot_view: RobotView

robot view for interfacing with joints / links / actuators

state_dim abstractmethod cached property

state_dim: int

sum of all the joints of interest, which defines the state of the robot.

action_dim abstractmethod

action_dim(move_group_ids: list[str]) -> int

sum of the commanded joints, which defines the action space of the robot. Args: move_group_ids: list of move group ids to consider for the action space

Source code in molmo_spaces/robots/abstract.py

@abc.abstractmethod
def action_dim(self, move_group_ids: list[str]) -> int:
    """sum of the commanded joints, which defines the action space of the robot.
    Args:
        move_group_ids: list of move group ids to consider for the action space
    """
    raise NotImplementedError

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, robot_spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations. Args: robot_config: The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.) spec: The scene to insert the robot into robot_spec: The robot model prefix: The prefix to use for the robot, i.e. the namespace pos: The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0. quat: The quaternion to use for the robot, in [w, x, y, z] format. randomize_textures: Whether to randomize the textures of the robot, if applicable. Not supported by all robots.

Source code in molmo_spaces/robots/abstract.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: "BaseRobotConfig",
    spec: MjSpec,
    robot_spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
) -> None:
    """
    Add a robot to a scene, taking care of any robot-specific considerations.
    Args:
        robot_config: The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)
        spec: The scene to insert the robot into
        robot_spec: The robot model
        prefix: The prefix to use for the robot, i.e. the namespace
        pos: The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.
        quat: The quaternion to use for the robot, in [w, x, y, z] format.
        randomize_textures: Whether to randomize the textures of the robot, if applicable. Not supported by all robots.
    """
    pos = pos + [0.0] if len(pos) == 2 else pos
    robot_root_name = cls.robot_model_root_name()
    attach_frame = spec.worldbody.add_frame(pos=pos, quat=quat)

    # Attach the robot to the base via the frame
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.exp_config.robot_config.action_noise_config
    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: "BaseRobotConfig"):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

compute_control abstractmethod

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/abstract.py

@abc.abstractmethod
def compute_control(self) -> None:
    """
    Compute and set the control inputs to the robot actuators based on the
    current state and the targets set by the user.
    Must be called after update_control().
    """
    raise NotImplementedError

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/abstract.py

def get_arm_move_group_ids(self) -> list[str]:
    """Get the move group IDs that should have TCP-bounded noise applied.

    Override in subclass to specify which move groups are arms.
    Default returns empty list (no noise applied).
    """
    return []

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset abstractmethod

reset() -> None

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/abstract.py

@abc.abstractmethod
def reset(self) -> None:
    """Reset the robot to its initial state or a provided state."""
    raise NotImplementedError

robot_model_root_name abstractmethod staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/abstract.py

@staticmethod
@abstractmethod
def robot_model_root_name() -> str:
    """The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion."""
    raise NotImplementedError

set_joint_pos abstractmethod

set_joint_pos(robot_joint_pos_dict) -> None

Set all the robot's joint positions to the specified values. Args: robot_joint_pos_dict: Dictionary containing joint positions for the robot based on the move groups ids.

Source code in molmo_spaces/robots/abstract.py

@abc.abstractmethod
def set_joint_pos(self, robot_joint_pos_dict) -> None:
    """Set all the robot's joint positions to the specified values.
    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
        based on the move groups ids.
    """
    raise NotImplementedError

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose abstractmethod

set_world_pose(robot_world_pose) -> None

Set the robot's world pose to the specified location (x-y-yaw) in the world.

Source code in molmo_spaces/robots/abstract.py

@abc.abstractmethod
def set_world_pose(self, robot_world_pose) -> None:
    """Set the robot's world pose to the specified location (x-y-yaw) in the world."""
    raise NotImplementedError

update_control abstractmethod

update_control(action_command_dict) -> None

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict		Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/abstract.py

@abc.abstractmethod
def update_control(self, action_command_dict) -> None:
    """
    Update the control targets to the robot based on the provided action commands.
    Does not set the control inputs to the robot actuators. See compute_control().

    Args:
        action_command_dict: Dictionary containing action commands for the robot
                             based on the move groups ids to be used.
    """
    raise NotImplementedError

bimanual_yam

Bimanual YAM robot implementation for the framework.

Classes:

Name	Description
BimanualYamRobot	Bimanual YAM robot implementation (two 6-DOF arms with parallel grippers).

BimanualYamRobot

BimanualYamRobot(mj_data: MjData, config: MlSpacesExpConfig)

Bases: Robot

Bimanual YAM robot implementation (two 6-DOF arms with parallel grippers).

Methods:

Name	Description
action_dim
add_robot_to_scene
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control
get_arm_move_group_ids	Bimanual YAM has two independent arms.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset
robot_model_root_name	The root body name in the bimanual_yam.xml.
set_joint_pos
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose
update_control

Attributes:

Name	Type	Description
controllers
exp_config
kinematics
mj_data
mj_model
namespace
parallel_kinematics
robot_view
state_dim	int

Source code in molmo_spaces/robots/bimanual_yam.py

def __init__(
    self,
    mj_data: MjData,
    config: "MlSpacesExpConfig",
) -> None:
    super().__init__(mj_data, config)
    self._robot_view = config.robot_config.robot_view_factory(
        mj_data, config.robot_config.robot_namespace
    )
    self._kinematics = BimanualYamKinematics(
        self.mj_model,
        namespace=config.robot_config.robot_namespace,
        robot_view_factory=config.robot_config.robot_view_factory,
    )

    # Use DummyParallelKinematics for batch IK (wraps the MlSpacesKinematics)
    # Default to left arm for parallel kinematics
    self._parallel_kinematics = DummyParallelKinematics(
        config.robot_config,
        self._kinematics,
        mg_id="left_arm",
        unlocked_mg_ids=["left_arm", "right_arm"],
    )

    # Determine controller classes based on command mode
    arm_command_mode = config.robot_config.command_mode.get("arm", "joint_position")
    if arm_command_mode == "joint_rel_position":
        arm_controller_cls = JointRelPosController
    else:
        arm_controller_cls = JointPosController

    gripper_command_mode = config.robot_config.command_mode.get("gripper", "joint_position")
    if gripper_command_mode == "joint_rel_position":
        gripper_controller_cls = JointRelPosController
    else:
        gripper_controller_cls = JointPosController

    self._controllers = {
        "left_arm": arm_controller_cls(self._robot_view.get_move_group("left_arm")),
        "right_arm": arm_controller_cls(self._robot_view.get_move_group("right_arm")),
        "left_gripper": gripper_controller_cls(self._robot_view.get_move_group("left_gripper")),
        "right_gripper": gripper_controller_cls(
            self._robot_view.get_move_group("right_gripper")
        ),
    }

controllers property

controllers

exp_config instance-attribute

exp_config = exp_config

kinematics property

kinematics

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

namespace property

namespace

parallel_kinematics property

parallel_kinematics

robot_view property

robot_view

state_dim property

state_dim: int

action_dim

action_dim(move_group_ids: list[str])

Source code in molmo_spaces/robots/bimanual_yam.py

def action_dim(self, move_group_ids: list[str]):
    return sum(self._robot_view.get_move_group(mg_id).n_actuators for mg_id in move_group_ids)

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BimanualYamRobotConfig, spec: MjSpec, robot_spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False) -> None

Source code in molmo_spaces/robots/bimanual_yam.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: "BimanualYamRobotConfig",
    spec: MjSpec,
    robot_spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
) -> None:
    robot_config = cast("BimanualYamRobotConfig", robot_config)
    add_base = robot_config.base_size is not None
    pos = pos + [0.0] if len(pos) == 2 else pos

    # Create a mocap body to control the robot base pose
    robot_body = spec.worldbody.add_body(
        name=f"{prefix}base",
        pos=pos,
        quat=quat,
        mocap=True,
    )

    if add_base:
        base_height = robot_config.base_size[2]

        # Create a base material (plain dark wood color)
        material_name = f"{prefix}robot_base_material"
        spec.add_material(name=material_name, rgba=[0.3, 0.2, 0.1, 1.0])

        # Add base geometry (platform)
        robot_body.add_geom(
            type=mjtGeom.mjGEOM_BOX,
            size=[x / 2 for x in robot_config.base_size],
            pos=[0, 0, base_height / 2],
            material=material_name,
            group=0,  # Visual group
        )
        attach_frame = robot_body.add_frame(pos=[0, 0, base_height])
    else:
        attach_frame = robot_body.add_frame()

    # Attach the bimanual robot model at the bimanual_base body
    robot_root_name = cls.robot_model_root_name()
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.exp_config.robot_config.action_noise_config
    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: "BaseRobotConfig"):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

compute_control

compute_control() -> None

Source code in molmo_spaces/robots/bimanual_yam.py

def compute_control(self) -> None:
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Bimanual YAM has two independent arms.

Source code in molmo_spaces/robots/bimanual_yam.py

def get_arm_move_group_ids(self) -> list[str]:
    """Bimanual YAM has two independent arms."""
    return ["left_arm", "right_arm"]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset() -> None

Source code in molmo_spaces/robots/bimanual_yam.py

def reset(self) -> None:
    for mg_id, default_pos in self.exp_config.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = default_pos

robot_model_root_name staticmethod

robot_model_root_name() -> str

The root body name in the bimanual_yam.xml.

Source code in molmo_spaces/robots/bimanual_yam.py

@staticmethod
def robot_model_root_name() -> str:
    """The root body name in the bimanual_yam.xml."""
    return "bimanual_base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict) -> None

Source code in molmo_spaces/robots/bimanual_yam.py

def set_joint_pos(self, robot_joint_pos_dict) -> None:
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self._robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose) -> None

Source code in molmo_spaces/robots/bimanual_yam.py

def set_world_pose(self, robot_world_pose) -> None:
    self._robot_view.base.pose = robot_world_pose

update_control

update_control(action_command_dict) -> None

Source code in molmo_spaces/robots/bimanual_yam.py

def update_control(self, action_command_dict) -> None:
    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

floating_robotiq

Classes:

Name	Description
FloatingRobotiqRobot

FloatingRobotiqRobot

FloatingRobotiqRobot(mj_data: MjData, config: MlSpacesExpConfig)

Bases: FloatingRUMRobot

Methods:

Name	Description
action_dim
add_robot_to_scene
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset
robot_model_root_name
set_joint_pos
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose
update_control

Attributes:

Name	Type	Description
controllers
exp_config
kinematics
mj_data
mj_model
namespace
parallel_kinematics
robot_view
state_dim

Source code in molmo_spaces/robots/floating_rum.py

def __init__(
    self,
    mj_data: MjData,
    config: "MlSpacesExpConfig",
    # robot_view_factory: RobotViewFactory = FloatingRUMRobotView,
):
    super().__init__(mj_data, config)
    self._robot_view = config.robot_config.robot_view_factory(
        mj_data, config.robot_config.robot_namespace
    )
    self._kinematics = FloatingRUMKinematics(
        self.mj_model,
        namespace=config.robot_config.robot_namespace,
        robot_view_factory=config.robot_config.robot_view_factory,
    )
    self._parallel_kinematics = DummyParallelKinematics(
        config.robot_config,
        self._kinematics,
        "gripper",
        ["base"],
    )
    self._last_cmd_action: dict[str, np.ndarray] | None = None

controllers property

controllers

exp_config instance-attribute

exp_config = exp_config

kinematics property

kinematics

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

namespace property

namespace

parallel_kinematics property

parallel_kinematics

robot_view property

robot_view

state_dim property

state_dim

action_dim

action_dim(move_group_ids: list[str])

Source code in molmo_spaces/robots/floating_rum.py

def action_dim(self, move_group_ids: list[str]):
    return sum(self._robot_view.get_move_group(mg_id).n_actuators for mg_id in move_group_ids)

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, robot_spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False) -> None

Source code in molmo_spaces/robots/floating_robotiq.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: "BaseRobotConfig",
    spec: MjSpec,
    robot_spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
) -> None:
    pos = pos + [0.0] if len(pos) == 2 else pos
    # call grandparent class method (Robot) directly, skipping FloatingRUMRobot
    # Use __func__ to get the unbound method and pass cls explicitly
    Robot.add_robot_to_scene.__func__(cls, robot_config, spec, robot_spec, prefix, pos, quat)

    # add target pose body and weld to base
    target_body_name = f"{prefix}target_ee_pose"
    spec.worldbody.add_body(name=target_body_name, pos=pos, quat=quat, mocap=True)
    eq = spec.add_equality()
    eq.name1 = target_body_name
    eq.name2 = f"{prefix}{cls.robot_model_root_name()}"
    # Stronger weld constraint: lower solref[0] = stiffer (0.001 is much stiffer than 0.02)
    # This makes the mocap body follow the target pose more precisely, allowing better lifting

    # best for lift to pick objects strongly
    eq.solref = np.array([0.001, 1])
    eq.solimp = np.array([0.99, 0.99, 0.001, 1, 2])

    # best for articulation where it's compliant``
    # eq.solref = np.array([0.01, 1])
    # eq.solimp = np.array([0.8, 0.8, 0.05, 1, 2])

    # in between?
    # eq.solref = np.array([0.0025, 1])
    # eq.solimp = np.array([0.99, 0.85, 0.01, 1, 2])

    eq.objtype = mjtObj.mjOBJ_BODY
    eq.type = mjtEq.mjEQ_WELD

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.exp_config.robot_config.action_noise_config
    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: "BaseRobotConfig"):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

compute_control

compute_control() -> None

Source code in molmo_spaces/robots/floating_rum.py

def compute_control(self) -> None:
    assert self._last_cmd_action is not None
    for mg_id, ctrl in self._last_cmd_action.items():
        if ctrl is not None:
            self._robot_view.get_move_group(mg_id).ctrl = ctrl

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/abstract.py

def get_arm_move_group_ids(self) -> list[str]:
    """Get the move group IDs that should have TCP-bounded noise applied.

    Override in subclass to specify which move groups are arms.
    Default returns empty list (no noise applied).
    """
    return []

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset()

Source code in molmo_spaces/robots/floating_rum.py

def reset(self):
    self._last_cmd_action = None
    for mg_id, default_pos in self.exp_config.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = default_pos

robot_model_root_name staticmethod

robot_model_root_name() -> str

Source code in molmo_spaces/robots/floating_rum.py

@staticmethod
def robot_model_root_name() -> str:
    return "base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict)

Source code in molmo_spaces/robots/floating_rum.py

def set_joint_pos(self, robot_joint_pos_dict):
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self._robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose)

Source code in molmo_spaces/robots/floating_rum.py

def set_world_pose(self, robot_world_pose):
    self._robot_view.base.pose = robot_world_pose

update_control

update_control(action_command_dict: dict[str, Any])

Source code in molmo_spaces/robots/floating_rum.py

def update_control(self, action_command_dict: dict[str, Any]):
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)
    self._last_cmd_action = action_command_dict

floating_rum

Classes:

Name	Description
FloatingRUMRobot	Floating RUM robot implementation for the framework.

FloatingRUMRobot

FloatingRUMRobot(mj_data: MjData, config: MlSpacesExpConfig)

Bases: Robot

Floating RUM robot implementation for the framework.

Methods:

Name	Description
action_dim
add_robot_to_scene
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset
robot_model_root_name
set_joint_pos
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose
update_control

Attributes:

Name	Type	Description
controllers
exp_config
kinematics
mj_data
mj_model
namespace
parallel_kinematics
robot_view
state_dim

Source code in molmo_spaces/robots/floating_rum.py

def __init__(
    self,
    mj_data: MjData,
    config: "MlSpacesExpConfig",
    # robot_view_factory: RobotViewFactory = FloatingRUMRobotView,
):
    super().__init__(mj_data, config)
    self._robot_view = config.robot_config.robot_view_factory(
        mj_data, config.robot_config.robot_namespace
    )
    self._kinematics = FloatingRUMKinematics(
        self.mj_model,
        namespace=config.robot_config.robot_namespace,
        robot_view_factory=config.robot_config.robot_view_factory,
    )
    self._parallel_kinematics = DummyParallelKinematics(
        config.robot_config,
        self._kinematics,
        "gripper",
        ["base"],
    )
    self._last_cmd_action: dict[str, np.ndarray] | None = None

controllers property

controllers

exp_config instance-attribute

exp_config = exp_config

kinematics property

kinematics

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

namespace property

namespace

parallel_kinematics property

parallel_kinematics

robot_view property

robot_view

state_dim property

state_dim

action_dim

action_dim(move_group_ids: list[str])

Source code in molmo_spaces/robots/floating_rum.py

def action_dim(self, move_group_ids: list[str]):
    return sum(self._robot_view.get_move_group(mg_id).n_actuators for mg_id in move_group_ids)

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, robot_spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False) -> None

Source code in molmo_spaces/robots/floating_rum.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: "BaseRobotConfig",
    spec: MjSpec,
    robot_spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
) -> None:
    pos = pos + [0.0] if len(pos) == 2 else pos
    super().add_robot_to_scene(
        robot_config, spec, robot_spec, prefix, pos, quat, randomize_textures
    )

    # add target pose body and weld to base
    target_body_name = f"{prefix}target_ee_pose"
    spec.worldbody.add_body(name=target_body_name, pos=pos, quat=quat, mocap=True)
    eq = spec.add_equality()
    eq.name1 = target_body_name
    eq.name2 = f"{prefix}{cls.robot_model_root_name()}"
    eq.solref = np.array([0.02, 1])
    eq.solimp = np.array([0.9, 0.95, 0.0, 1, 2])
    eq.objtype = mjtObj.mjOBJ_BODY
    eq.type = mjtEq.mjEQ_WELD

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.exp_config.robot_config.action_noise_config
    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: "BaseRobotConfig"):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

compute_control

compute_control() -> None

Source code in molmo_spaces/robots/floating_rum.py

def compute_control(self) -> None:
    assert self._last_cmd_action is not None
    for mg_id, ctrl in self._last_cmd_action.items():
        if ctrl is not None:
            self._robot_view.get_move_group(mg_id).ctrl = ctrl

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/abstract.py

def get_arm_move_group_ids(self) -> list[str]:
    """Get the move group IDs that should have TCP-bounded noise applied.

    Override in subclass to specify which move groups are arms.
    Default returns empty list (no noise applied).
    """
    return []

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset()

Source code in molmo_spaces/robots/floating_rum.py

def reset(self):
    self._last_cmd_action = None
    for mg_id, default_pos in self.exp_config.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = default_pos

robot_model_root_name staticmethod

robot_model_root_name() -> str

Source code in molmo_spaces/robots/floating_rum.py

@staticmethod
def robot_model_root_name() -> str:
    return "base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict)

Source code in molmo_spaces/robots/floating_rum.py

def set_joint_pos(self, robot_joint_pos_dict):
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self._robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose)

Source code in molmo_spaces/robots/floating_rum.py

def set_world_pose(self, robot_world_pose):
    self._robot_view.base.pose = robot_world_pose

update_control

update_control(action_command_dict: dict[str, Any])

Source code in molmo_spaces/robots/floating_rum.py

def update_control(self, action_command_dict: dict[str, Any]):
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)
    self._last_cmd_action = action_command_dict

franka

Classes:

Name	Description
FrankaRobot	Franka robot implementation for the framework.

Attributes:

Name	Type	Description
log

log module-attribute

log = getLogger(__name__)

FrankaRobot

FrankaRobot(mj_data: MjData, config: MlSpacesExpConfig)

Bases: Robot

Franka robot implementation for the framework.

Methods:

Name	Description
action_dim
add_robot_to_scene
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control
create_robot_base_material
get_arm_move_group_ids	Franka has a single arm move group.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
randomize_robot_textures
reset
robot_model_root_name
set_joint_pos
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose
update_control

Attributes:

Name	Type	Description
controllers	dict[str, Controller]
exp_config
kinematics
mj_data
mj_model
namespace
parallel_kinematics
robot_view
state_dim	int

Source code in molmo_spaces/robots/franka.py

def __init__(
    self,
    mj_data: MjData,
    config: "MlSpacesExpConfig",
) -> None:
    super().__init__(mj_data, config)
    self._robot_view = config.robot_config.robot_view_factory(
        mj_data, config.robot_config.robot_namespace
    )
    self._kinematics = FrankaKinematics(
        self.mj_model,
        namespace=config.robot_config.robot_namespace,
        robot_view_factory=config.robot_config.robot_view_factory,
    )

    self._parallel_kinematics = FrankaParallelKinematics(config.robot_config)
    arm_controller_cls = (
        JointPosController
        if config.robot_config.command_mode == {}
        or config.robot_config.command_mode["arm"] == "joint_position"
        else JointRelPosController
    )
    self._controllers = {
        "arm": arm_controller_cls(self._robot_view.get_move_group("arm")),
        "gripper": JointPosController(self._robot_view.get_move_group("gripper")),
    }

controllers property

controllers: dict[str, Controller]

exp_config instance-attribute

exp_config = exp_config

kinematics property

kinematics

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

namespace property

namespace

parallel_kinematics property

parallel_kinematics

robot_view property

robot_view

state_dim property

state_dim: int

action_dim

action_dim(move_group_ids: list[str])

Source code in molmo_spaces/robots/franka.py

def action_dim(self, move_group_ids: list[str]):
    return sum(self._robot_view.get_move_group(mg_id).n_actuators for mg_id in move_group_ids)

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: FrankaRobotConfig, spec: MjSpec, robot_spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False) -> None

Source code in molmo_spaces/robots/franka.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: "FrankaRobotConfig",
    spec: MjSpec,
    robot_spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
) -> None:
    robot_config = cast("FrankaRobotConfig", robot_config)
    add_base = robot_config.base_size is not None
    pos = pos + [0.0] if len(pos) == 2 else pos

    material_name = cls.create_robot_base_material(
        robot_config, spec, prefix, randomize_textures
    )

    robot_body = spec.worldbody.add_body(
        name=f"{prefix}base",
        pos=pos,
        quat=quat,
        mocap=True,
    )
    if add_base:
        base_height = robot_config.base_size[2]

        # Add base geometry (wooden platform)
        robot_body.add_geom(
            type=mjtGeom.mjGEOM_BOX,
            size=[x / 2 for x in robot_config.base_size],
            pos=[0, 0, base_height / 2],
            material=material_name,
            group=0,  # Visual group
        )
        attach_frame = robot_body.add_frame(pos=[0, 0, base_height])
    else:
        attach_frame = robot_body.add_frame()

    if randomize_textures:
        cls.randomize_robot_textures(robot_config, spec, prefix, robot_spec)

    # Attach the robot to the base via the frame
    robot_root_name = cls.robot_model_root_name()
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.exp_config.robot_config.action_noise_config
    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: "BaseRobotConfig"):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

compute_control

compute_control() -> None

Source code in molmo_spaces/robots/franka.py

def compute_control(self) -> None:
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

create_robot_base_material classmethod

create_robot_base_material(robot_config: FrankaRobotConfig, spec: MjSpec, prefix: str, randomize_base_texture: bool) -> None

Source code in molmo_spaces/robots/franka.py

@classmethod
def create_robot_base_material(
    cls,
    robot_config: "FrankaRobotConfig",
    spec: MjSpec,
    prefix: str,
    randomize_base_texture: bool,
) -> None:
    texture_dir = get_robot_path(robot_config.name) / "assets" / "base_textures"
    assert texture_dir.is_dir(), f"Texture directory {texture_dir} does not exist"
    texture_path: Path | None = None
    if randomize_base_texture:
        texture_paths = list(texture_dir.glob("*.png"))
        texture_paths.sort(key=lambda x: x.name)
        assert len(texture_paths) > 0, f"No robot base texture paths found in {texture_dir}"
        log.debug(f"Found {len(texture_paths)} robot base texture paths")
        texture_path = random.choice(texture_paths)
    else:
        texture_path = texture_dir / "DarkWood2.png"
        assert texture_path.is_file(), f"Default texture {texture_path} does not exist"

    texture_name = f"{prefix}robot_base_texture"
    spec.add_texture(
        name=texture_name,
        type=mujoco.mjtTexture.mjTEXTURE_CUBE,
        file=str(texture_path),
    )
    log.debug(f"Successfully created texture from {texture_path}")

    material_name = f"{prefix}robot_base_material"
    robot_base_mat = spec.add_material(name=material_name)
    robot_base_mat.textures[mujoco.mjtTextureRole.mjTEXROLE_RGB] = texture_name
    log.debug(f"Successfully created material {material_name}")
    return material_name

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Franka has a single arm move group.

Source code in molmo_spaces/robots/franka.py

def get_arm_move_group_ids(self) -> list[str]:
    """Franka has a single arm move group."""
    return ["arm"]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

randomize_robot_textures classmethod

randomize_robot_textures(robot_config: FrankaRobotConfig, spec: MjSpec, prefix: str, robot_spec: MjSpec)

Source code in molmo_spaces/robots/franka.py

@classmethod
def randomize_robot_textures(
    cls,
    robot_config: "FrankaRobotConfig",
    spec: MjSpec,
    prefix: str,
    robot_spec: MjSpec,
):
    if random.random() > robot_config.perturb_texture_probability:
        log.info(f"Skipping texture randomization for robot '{robot_config.name}'")
        return

    perturbed_materials: dict[str, str] = {}
    for material in robot_spec.materials:
        material: mujoco.MjsMaterial
        is_rgb_mat = all(
            material.textures[i] == "" for i in range(mujoco.mjtTextureRole.mjNTEXROLE)
        )
        if not is_rgb_mat:
            continue

        speckle_img = _speckle_texture(material.rgba[:3])
        buffer = BytesIO()
        speckle_img.save(buffer, format="PNG")
        buffer.seek(0)

        tex_name = f"{material.name}_perturbed_tex"
        mat_name = f"{material.name}_perturbed"
        fn = f"{prefix}{tex_name}.png".replace("/", "__")
        spec.assets[fn] = buffer.getvalue()
        robot_spec.add_texture(name=tex_name, type=mujoco.mjtTexture.mjTEXTURE_2D, file=fn)
        perturbed_mat = robot_spec.add_material(name=mat_name)
        perturbed_mat.textures[mujoco.mjtTextureRole.mjTEXROLE_RGB] = tex_name
        perturbed_materials[material.name] = mat_name

    def set_material(body: MjsBody):
        for geom in body.geoms:
            geom: mujoco.MjsGeom
            if geom.material in perturbed_materials:
                log.debug(
                    f"Setting material {geom.material} to {perturbed_materials[geom.material]} "
                    f"for geom '{geom.name}' in body '{body.name}'"
                )
                geom.material = perturbed_materials[geom.material]
        for child in body.bodies:
            set_material(child)

    robot_body = robot_spec.body(cls.robot_model_root_name())
    set_material(robot_body)
    log.info(f"Successfully randomized robot textures for robot '{robot_config.name}'")

reset

reset() -> None

Source code in molmo_spaces/robots/franka.py

def reset(self) -> None:
    for mg_id, default_pos in self.exp_config.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = default_pos

robot_model_root_name staticmethod

robot_model_root_name() -> str

Source code in molmo_spaces/robots/franka.py

@staticmethod
def robot_model_root_name() -> str:
    return "fr3_link0"

set_joint_pos

set_joint_pos(robot_joint_pos_dict) -> None

Source code in molmo_spaces/robots/franka.py

def set_joint_pos(self, robot_joint_pos_dict) -> None:
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self._robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose) -> None

Source code in molmo_spaces/robots/franka.py

def set_world_pose(self, robot_world_pose) -> None:
    self._robot_view.base.pose = robot_world_pose

update_control

update_control(action_command_dict: dict[str, Any]) -> None

Source code in molmo_spaces/robots/franka.py

def update_control(self, action_command_dict: dict[str, Any]) -> None:
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

i2rt_yam

i2rt YAM robot implementation for the framework.

Classes:

Name	Description
I2rtYamRobot	i2rt YAM 6-DOF arm robot implementation.

I2rtYamRobot

I2rtYamRobot(mj_data: MjData, config: MlSpacesExpConfig)

Bases: Robot

i2rt YAM 6-DOF arm robot implementation.

Methods:

Name	Description
action_dim
add_robot_to_scene
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control
get_arm_move_group_ids	YAM has a single arm move group.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset
robot_model_root_name
set_joint_pos
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose
update_control

Attributes:

Name	Type	Description
controllers
exp_config
kinematics
mj_data
mj_model
namespace
parallel_kinematics
robot_view
state_dim	int

Source code in molmo_spaces/robots/i2rt_yam.py

def __init__(
    self,
    mj_data: MjData,
    config: "MlSpacesExpConfig",
) -> None:
    super().__init__(mj_data, config)
    self._robot_view = config.robot_config.robot_view_factory(
        mj_data, config.robot_config.robot_namespace
    )
    self._kinematics = I2rtYamKinematics(
        self.mj_model,
        namespace=config.robot_config.robot_namespace,
        robot_view_factory=config.robot_config.robot_view_factory,
    )

    # Use DummyParallelKinematics for batch IK (wraps the MlSpacesKinematics)
    self._parallel_kinematics = DummyParallelKinematics(
        config.robot_config,
        self._kinematics,
        mg_id="arm",
        unlocked_mg_ids=["arm"],
    )

    arm_controller_cls = (
        JointPosController
        if config.robot_config.command_mode == {}
        or config.robot_config.command_mode.get("arm") == "joint_position"
        else JointRelPosController
    )
    self._controllers = {
        "arm": arm_controller_cls(self._robot_view.get_move_group("arm")),
        "gripper": JointPosController(self._robot_view.get_move_group("gripper")),
    }

controllers property

controllers

exp_config instance-attribute

exp_config = exp_config

kinematics property

kinematics

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

namespace property

namespace

parallel_kinematics property

parallel_kinematics

robot_view property

robot_view

state_dim property

state_dim: int

action_dim

action_dim(move_group_ids: list[str])

Source code in molmo_spaces/robots/i2rt_yam.py

def action_dim(self, move_group_ids: list[str]):
    return sum(self._robot_view.get_move_group(mg_id).n_actuators for mg_id in move_group_ids)

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: I2rtYamRobotConfig, spec: MjSpec, robot_spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False) -> None

Source code in molmo_spaces/robots/i2rt_yam.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: "I2rtYamRobotConfig",
    spec: MjSpec,
    robot_spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
) -> None:
    robot_config = cast("I2rtYamRobotConfig", robot_config)
    add_base = robot_config.base_size is not None
    pos = pos + [0.0] if len(pos) == 2 else pos

    # Create a mocap body to control the robot base pose
    robot_body = spec.worldbody.add_body(
        name=f"{prefix}base",
        pos=pos,
        quat=quat,
        mocap=True,
    )

    if add_base:
        base_height = robot_config.base_size[2]

        # Create a base material (plain dark wood color)
        material_name = f"{prefix}robot_base_material"
        spec.add_material(name=material_name, rgba=[0.3, 0.2, 0.1, 1.0])

        # Add base geometry (platform)
        robot_body.add_geom(
            type=mjtGeom.mjGEOM_BOX,
            size=[x / 2 for x in robot_config.base_size],
            pos=[0, 0, base_height / 2],
            material=material_name,
            group=0,  # Visual group
        )
        attach_frame = robot_body.add_frame(pos=[0, 0, base_height])
    else:
        attach_frame = robot_body.add_frame()

    # Attach the robot to the mocap body (on top of platform if present)
    robot_root_name = cls.robot_model_root_name()
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.exp_config.robot_config.action_noise_config
    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: "BaseRobotConfig"):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

compute_control

compute_control() -> None

Source code in molmo_spaces/robots/i2rt_yam.py

def compute_control(self) -> None:
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

YAM has a single arm move group.

Source code in molmo_spaces/robots/i2rt_yam.py

def get_arm_move_group_ids(self) -> list[str]:
    """YAM has a single arm move group."""
    return ["arm"]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset() -> None

Source code in molmo_spaces/robots/i2rt_yam.py

def reset(self) -> None:
    for mg_id, default_pos in self.exp_config.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = default_pos

robot_model_root_name staticmethod

robot_model_root_name() -> str

Source code in molmo_spaces/robots/i2rt_yam.py

@staticmethod
def robot_model_root_name() -> str:
    return "arm"

set_joint_pos

set_joint_pos(robot_joint_pos_dict) -> None

Source code in molmo_spaces/robots/i2rt_yam.py

def set_joint_pos(self, robot_joint_pos_dict) -> None:
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self._robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose) -> None

Source code in molmo_spaces/robots/i2rt_yam.py

def set_world_pose(self, robot_world_pose) -> None:
    self._robot_view.base.pose = robot_world_pose

update_control

update_control(action_command_dict) -> None

Source code in molmo_spaces/robots/i2rt_yam.py

def update_control(self, action_command_dict) -> None:
    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

rby1

Classes:

Name	Description
RBY1	RBY1 Robot class for the RBY1 robot.

RBY1

RBY1(mj_data: MjData, exp_config: MlSpacesExpConfig)

Bases: Robot

RBY1 Robot class for the RBY1 robot. This class extends the Robot class and provides specific implementations for the RBY1 robot including the robot view, controllers, and a kinematic solver.

The mode in which the robot arm, gripper, or base is commanded (Eg. ee position command or joint position command) can be set using the arm_command_mode, gripper_command_mode, and base_command_mode parameters in the robot configuration.

The move_group i.e., the set of robot parts that are to be moved can be dynamically changed, e.g., move base for navigation, then move both base and arm for opening doors.

Parameters:

Name	Type	Description	Default
exp_config	MlSpacesExpConfig	Experiment configuration params	required
mj_data	MjData	MuJoCo data structure containing the current simulation state	required

Methods:

Name	Description
action_dim
add_robot_to_scene
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
check_collision
compute_control
get_arm_move_group_ids	RBY1 has two independent arms - each gets independent noise.
get_grasped_objs	Return a set of grasped object body IDs.
get_joint_position	Get the current joint positions of the move groups
get_joint_ranges	Get the joint ranges of the move groups
get_world_pose_tf_mat	Get the robot's world pose transformation matrix.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial position or a provided set of positions and world pose.
robot_model_root_name
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified location in the world.
update_control	Update the control inputs to the robot based on the provided action commands.

Attributes:

Name	Type	Description
arm_command_mode
arm_command_modes
base_command_mode
base_command_modes
controllers	dict[str, Controller]
exp_config
gripper_command_mode
gripper_command_modes
head_command_mode
kinematics
mj_data
mj_model
namespace
parallel_kinematics
robot_view
state_dim	int
torso_command_mode
torso_command_modes

Source code in molmo_spaces/robots/rby1.py

def __init__(self, mj_data: MjData, exp_config: "MlSpacesExpConfig") -> None:
    """
    Args:
        exp_config: Experiment configuration params
        mj_data: MuJoCo data structure containing the current simulation state
    """
    super().__init__(mj_data, exp_config)

    self._namespace = self.exp_config.robot_config.robot_namespace
    self._use_holo_base = self.exp_config.robot_config.use_holo_base

    # Create the robot view:
    self._robot_view = RBY1RobotView(mj_data, self.namespace, holo_base=self._use_holo_base)

    # Create kinematic solver:
    self._kinematics = RBY1Kinematics(
        self.mj_model, namespace=self.namespace, holo_base=self._use_holo_base
    )

    # Create controllers:

    # All the joint actuators are position controlled.
    # However, they can be commanded in other modes like velocity, ee position, etc. and the controller will
    # perform the necessary conversions to joint position.
    self.arm_command_modes = [
        "joint_position",
        "joint_rel_position",
        "joint_velocity",
        "ee_position",
        "ee_velocity",
    ]
    if self.exp_config.robot_config.command_mode["arm"] is not None:
        assert self.exp_config.robot_config.command_mode["arm"] in self.arm_command_modes, (
            f"Arm command mode {self.exp_config.robot_config.command_mode['arm']} not in {self.arm_command_modes}"
        )
    self.arm_command_mode = (
        "joint_position"
        if not self.exp_config.robot_config.command_mode["arm"]
        else self.exp_config.robot_config.command_mode["arm"]
    )
    if self.arm_command_mode == "joint_rel_position":
        left_arm_controller = JointRelPosController(self.robot_view.get_move_group("left_arm"))
        right_arm_controller = JointRelPosController(
            self.robot_view.get_move_group("right_arm")
        )
    elif self.arm_command_mode == "joint_position":
        left_arm_controller = JointPosController(self.robot_view.get_move_group("left_arm"))
        right_arm_controller = JointPosController(self.robot_view.get_move_group("right_arm"))
    else:
        raise NotImplementedError(
            f"Arm command mode {self.arm_command_mode} not implemented yet."
        )

    # Gripper command modes - separate from arm command modes
    self.gripper_command_modes = [
        "joint_position",
        "joint_rel_position",
        "joint_velocity",
    ]
    if self.exp_config.robot_config.command_mode["gripper"] is not None:
        assert (
            self.exp_config.robot_config.command_mode["gripper"] in self.gripper_command_modes
        ), (
            f"Gripper command mode {self.exp_config.robot_config.command_mode['gripper']} not in {self.gripper_command_modes}"
        )
    self.gripper_command_mode = (
        "joint_position"
        if not self.exp_config.robot_config.command_mode["gripper"]
        else self.exp_config.robot_config.command_mode["gripper"]
    )
    if self.gripper_command_mode == "joint_rel_position":
        left_gripper_controller = JointRelPosController(
            self.robot_view.get_move_group("left_gripper")
        )
        right_gripper_controller = JointRelPosController(
            self.robot_view.get_move_group("right_gripper")
        )
    elif self.gripper_command_mode == "joint_position":
        left_gripper_controller = JointPosController(
            self.robot_view.get_move_group("left_gripper")
        )
        right_gripper_controller = JointPosController(
            self.robot_view.get_move_group("right_gripper")
        )
    else:
        raise NotImplementedError(
            f"Gripper command mode {self.gripper_command_mode} not implemented yet."
        )

    # The base actuators (wheels) are usually velocity controlled. If using virtual holonomic joints, they are position controlled.
    # Base actuators can still be commanded in other modes like planar position, planar velocity, etc.
    # and the controller should perform the necessary conversions to wheel velocity or holo joint position.
    self.base_command_modes = [
        "planar_position",
        "planar_velocity",
        "wheel_velocity",
        "holo_joint_planar_position",
        "holo_joint_rel_planar_position",
    ]
    if self.exp_config.robot_config.command_mode["base"] is not None:
        assert self.exp_config.robot_config.command_mode["base"] in self.base_command_modes, (
            f"Base command mode {self.exp_config.robot_config.command_mode['base']} not in {self.base_command_modes}"
        )
    self.base_command_mode = (
        "planar_position"
        if not self.exp_config.robot_config.command_mode["base"]
        else self.exp_config.robot_config.command_mode["base"]
    )
    if self.base_command_mode == "planar_position":
        base_controller = DiffDriveBasePoseController(
            self.exp_config.robot_config, self.robot_view.get_move_group("base")
        )
    elif self.base_command_mode == "holo_joint_rel_planar_position":
        base_controller = JointRelPosController(self.robot_view.get_move_group("base"))
    elif self.base_command_mode == "holo_joint_planar_position":
        base_controller = JointPosController(self.robot_view.get_move_group("base"))
    else:
        raise NotImplementedError(
            f"Base command mode {self.base_command_mode} not implemented yet."
        )

    # Head is fixed - no head actions are supported
    self.head_command_mode = self.exp_config.robot_config.command_mode.get("head")
    assert self.head_command_mode is None, (
        "RBY1 head actuation is disabled. The head is fixed at init_qpos['head'] with optional "
        "randomization via init_qpos_noise_range['head']. "
        "Do not set command_mode['head'] to a non-None value."
    )

    # Torso command modes
    self.torso_command_modes = ["joint_position", "height"]
    self.torso_command_mode = (
        self.exp_config.robot_config.command_mode.get("torso", "joint_position")
        or "joint_position"
    )
    assert self.torso_command_mode in self.torso_command_modes, (
        f"Torso command mode {self.torso_command_mode} not in {self.torso_command_modes}"
    )
    if self.torso_command_mode == "height":
        torso_controller = TorsoHeightJointPosController(
            self.robot_view.get_move_group("torso")
        )
    else:
        torso_controller = JointPosController(self.robot_view.get_move_group("torso"))

    self._controllers = {
        "base": base_controller,
        "torso": torso_controller,
        "left_arm": left_arm_controller,
        "right_arm": right_arm_controller,
        "left_gripper": left_gripper_controller,
        "right_gripper": right_gripper_controller,
    }
    assert set(self._controllers.keys()).issubset(set(self._robot_view.move_group_ids())), (
        "All controller keys must be move group IDs"
    )

arm_command_mode instance-attribute

arm_command_mode = 'joint_position' if not command_mode['arm'] else command_mode['arm']

arm_command_modes instance-attribute

arm_command_modes = ['joint_position', 'joint_rel_position', 'joint_velocity', 'ee_position', 'ee_velocity']

base_command_mode instance-attribute

base_command_mode = 'planar_position' if not command_mode['base'] else command_mode['base']

base_command_modes instance-attribute

base_command_modes = ['planar_position', 'planar_velocity', 'wheel_velocity', 'holo_joint_planar_position', 'holo_joint_rel_planar_position']

controllers property

controllers: dict[str, Controller]

exp_config instance-attribute

exp_config = exp_config

gripper_command_mode instance-attribute

gripper_command_mode = 'joint_position' if not command_mode['gripper'] else command_mode['gripper']

gripper_command_modes instance-attribute

gripper_command_modes = ['joint_position', 'joint_rel_position', 'joint_velocity']

head_command_mode instance-attribute

head_command_mode = get('head')

kinematics property

kinematics

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

namespace property

namespace

parallel_kinematics property

parallel_kinematics

robot_view property

robot_view

state_dim cached property

state_dim: int

torso_command_mode instance-attribute

torso_command_mode = get('torso', 'joint_position') or 'joint_position'

torso_command_modes instance-attribute

torso_command_modes = ['joint_position', 'height']

action_dim

action_dim(move_group_ids: list) -> int

Source code in molmo_spaces/robots/rby1.py

def action_dim(self, move_group_ids: list) -> int:
    # return sum of the commanded joints based on the move group ids
    action_dim = 0
    if "base" in move_group_ids:
        if "planar" in self.base_command_mode:
            action_dim += 3  # actions provided are 2D planar positions / velocities
        else:
            action_dim += self._robot_view.get_move_group(
                "base"
            ).n_actuators  # wheel velocities
    if "torso" in move_group_ids:
        if self.torso_command_mode == "height":
            action_dim += 1
        else:
            action_dim += self._robot_view.get_move_group("torso").n_actuators
    if "left_arm" in move_group_ids:
        if "joint" in self.arm_command_mode:
            action_dim += self._robot_view.get_move_group("left_arm").n_actuators
        elif "ee" in self.arm_command_mode:
            action_dim += 7  # ee pos + orientation (quaternion)
    if "right_arm" in move_group_ids:
        if "joint" in self.arm_command_mode:
            action_dim += self._robot_view.get_move_group("right_arm").n_actuators
        elif "ee" in self.arm_command_mode:
            action_dim += 7  # ee pos + orientation (quaternion)
    if "left_gripper" in move_group_ids:
        action_dim += self._robot_view.get_move_group("left_gripper").n_actuators
    if "right_gripper" in move_group_ids:
        action_dim += self._robot_view.get_move_group("right_gripper").n_actuators
    # Note: head is not included - RBY1 head actuation is disabled
    return action_dim

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: RobotConfig, spec: MjSpec, robot_spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False) -> None

Source code in molmo_spaces/robots/rby1.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: "MlSpacesExpConfig.RobotConfig",
    spec: MjSpec,
    robot_spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
) -> None:
    super().add_robot_to_scene(
        robot_config, spec, robot_spec, prefix, pos, quat, randomize_textures
    )

    def add_slider_act(
        name: str, ctrlrange: float, gainprm: float, biasprm: list[float], gear_idx: int
    ):
        act = spec.add_actuator()
        act.name = f"{prefix}{name}"
        act.target = f"{prefix}base_site"
        act.refsite = f"{prefix}world"
        act.ctrlrange = np.array([-ctrlrange, ctrlrange])
        act.gainprm[0] = gainprm
        act.biasprm[: len(biasprm)] = biasprm
        act.trntype = mujoco.mjtTrn.mjTRN_SITE
        act.biastype = mujoco.mjtBias.mjBIAS_AFFINE
        gear = [0] * 6
        gear[gear_idx] = 1
        act.gear = gear
        return act

    if robot_config.use_holo_base:
        spec.worldbody.add_site(name=f"{prefix}world", pos=[0, 0, 0.005], quat=[1, 0, 0, 0])
        add_slider_act("base_x_act", 25, 25000, [0, -25000, 0.5], 0)
        add_slider_act("base_y_act", 25, 25000, [0, -25000, 0.5], 1)
        add_slider_act("base_theta_act", np.pi, 5000, [0, -5000, 0.5], 5)

    # TODO(snehal): don't use bodies in the MJCF, just use visual geoms to render these
    # add target ee pose bodies
    ee_viz_right = spec.worldbody.add_body(
        name="target_ee_pose_right", pos=[0, 0, 0], quat=[1, 0, 0, 0], mocap=True
    )
    ee_viz_right.add_site(
        name="target_ee_pose_right",
        type=mujoco.mjtGeom.mjGEOM_BOX,
        size=[0.05, 0.05, 0.05],
        rgba=[0, 0, 1, 0.3],
        group=1,
    )

    ee_viz_left = spec.worldbody.add_body(
        name="target_ee_pose_left", pos=[0, 0, 0], quat=[1, 0, 0, 0], mocap=True
    )
    ee_viz_left.add_site(
        name="target_ee_pose_left",
        type=mujoco.mjtGeom.mjGEOM_BOX,
        size=[0.05, 0.05, 0.05],
        rgba=[1, 0, 0, 0.3],
        group=1,
    )

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Extends the base class implementation to also apply base noise for RBY1's holonomic base commands.

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/rby1.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Extends the base class implementation to also apply base noise
    for RBY1's holonomic base commands.

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.exp_config.robot_config.action_noise_config
    if not noise_config.enabled:
        return action

    # Apply arm noise via parent class
    noisy_action = super().apply_action_noise(action)

    # Apply base noise if base command is present and using holonomic planar mode
    if "base" in action and action["base"] is not None:
        if "holo_joint" in self.base_command_mode and "planar" in self.base_command_mode:
            commanded_base_pos = np.asarray(action["base"])
            noisy_base_pos = self._apply_base_noise(commanded_base_pos)
            noisy_action["base"] = noisy_base_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: "BaseRobotConfig"):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

check_collision

check_collision(model: MjModel, data: MjData, grasped_objs: set[int] = None) -> bool

Source code in molmo_spaces/robots/rby1.py

def check_collision(self, model: MjModel, data: MjData, grasped_objs: set[int] = None) -> bool:
    # TODO
    pass

compute_control

compute_control() -> None

Source code in molmo_spaces/robots/rby1.py

def compute_control(self) -> None:
    for mg_id, controller in self.controllers.items():
        ctrl_inputs = controller.compute_ctrl_inputs()
        self.robot_view.get_move_group(mg_id).ctrl = ctrl_inputs

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

RBY1 has two independent arms - each gets independent noise.

Source code in molmo_spaces/robots/rby1.py

def get_arm_move_group_ids(self) -> list[str]:
    """RBY1 has two independent arms - each gets independent noise."""
    return ["left_arm", "right_arm"]

get_grasped_objs

get_grasped_objs(model: MjModel, data: MjData) -> set[int]

Return a set of grasped object body IDs.

Source code in molmo_spaces/robots/rby1.py

def get_grasped_objs(self, model: MjModel, data: MjData) -> set[int]:
    """Return a set of grasped object body IDs."""
    # TODO
    pass

get_joint_position

get_joint_position(move_group_ids: list[str]) -> ndarray

Get the current joint positions of the move groups

Source code in molmo_spaces/robots/rby1.py

def get_joint_position(self, move_group_ids: list[str]) -> np.ndarray:
    """Get the current joint positions of the move groups"""
    return np.concatenate(
        [
            self._robot_view.get_move_group(move_group_id).joint_pos.copy()
            for move_group_id in move_group_ids
        ]
    )

get_joint_ranges

get_joint_ranges(move_group_ids: list[str])

Get the joint ranges of the move groups

Source code in molmo_spaces/robots/rby1.py

def get_joint_ranges(self, move_group_ids: list[str]):
    """Get the joint ranges of the move groups"""
    joint_ranges = {}
    count = 0
    for move_group_id in move_group_ids:
        joint_ranges[move_group_id] = (
            count,
            count + self._robot_view.get_move_group(move_group_id).n_joints,
        )
        count += self._robot_view.get_move_group(move_group_id).n_joints
    return joint_ranges

get_world_pose_tf_mat

get_world_pose_tf_mat()

Get the robot's world pose transformation matrix. Returns: np.ndarray: 4x4 transformation matrix for the robot base pose in world frame

Source code in molmo_spaces/robots/rby1.py

def get_world_pose_tf_mat(self):
    """Get the robot's world pose transformation matrix.
    Returns:
        np.ndarray: 4x4 transformation matrix for the robot base pose in world frame
    """
    return self.robot_view.get_move_group("base").pose

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset(robot_joint_pos_dict=None, robot_world_pose=None) -> None

Reset the robot to its initial position or a provided set of positions and world pose.

Source code in molmo_spaces/robots/rby1.py

def reset(self, robot_joint_pos_dict=None, robot_world_pose=None) -> None:
    """Reset the robot to its initial position or a provided set of positions and world pose."""

    # Load default robot configuration and world pose
    init_qpos_dict = self.exp_config.robot_config.init_qpos
    default_world_pose = self.exp_config.robot_config.default_world_pose
    if robot_joint_pos_dict is not None:
        for move_group_id, joint_pos in robot_joint_pos_dict.items():
            init_qpos_dict[move_group_id] = joint_pos
    if robot_world_pose is not None:
        default_world_pose = robot_world_pose
    # Set the joint positions
    self.set_joint_pos(init_qpos_dict)
    # Set the world pose
    self.set_world_pose(default_world_pose)

    # reset controllers
    for _, controller in self._controllers.items():
        controller.reset()

    # Set the head ctrl to match its qpos to prevent jerking at trajectory start.
    # The head has actuators but no controller, so we manually set ctrl = noop_ctrl.
    head_mg = self.robot_view.get_move_group("head")
    head_mg.ctrl = head_mg.noop_ctrl

robot_model_root_name staticmethod

robot_model_root_name() -> str

Source code in molmo_spaces/robots/rby1.py

@staticmethod
def robot_model_root_name() -> str:
    return "base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict) -> None

Set all the robot's joint positions to the specified values. Args: robot_joint_pos_dict: Dictionary or SimpleNamespace containing joint positions for the robot based on the move groups ids.

Source code in molmo_spaces/robots/rby1.py

def set_joint_pos(self, robot_joint_pos_dict) -> None:
    """Set all the robot's joint positions to the specified values.
    Args:
        robot_joint_pos_dict: Dictionary or SimpleNamespace containing joint positions for the robot
        based on the move groups ids.
    """
    # Handle both dict and SimpleNamespace objects
    if hasattr(robot_joint_pos_dict, "__dict__"):
        # SimpleNamespace object
        items = robot_joint_pos_dict.__dict__.items()
    else:
        # Dictionary object
        items = robot_joint_pos_dict.items()

    for move_group_id, joint_pos in items:
        if move_group_id in self.robot_view.move_group_ids():
            move_group = self.robot_view.get_move_group(move_group_id)
            # set the joint positions
            move_group.joint_pos = joint_pos
        else:
            raise ValueError(f"Move group {move_group_id} not found in robot view.")

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose) -> None

Set the robot's world pose to the specified location in the world.

Source code in molmo_spaces/robots/rby1.py

def set_world_pose(self, robot_world_pose) -> None:
    """Set the robot's world pose to the specified location in the world."""

    pose_tf = np.eye(4, dtype=np.float64)
    if len(robot_world_pose) == 3:
        # (x, y, theta)
        x, y, theta = robot_world_pose
        pose_tf[:2, :2] = np.array(
            [
                [np.cos(theta), -np.sin(theta)],
                [np.sin(theta), np.cos(theta)],
            ]
        )
        pose_tf[:3, 3] = np.array([x, y, 0.0])
    elif len(robot_world_pose) == 7:
        # (x, y, z, w, x, y, z) (pos + quat, scalar_first)
        pos = np.array(robot_world_pose[:3], dtype=np.float64)
        quat = np.array(robot_world_pose[3:], dtype=np.float64)
        rot_mat = R.from_quat(quat, scalar_first=True).as_matrix()
        pose_tf[:3, :3] = rot_mat
        pose_tf[:3, 3] = pos
    else:
        raise ValueError(
            "robot_world_pose must be either (x, y, theta) or (x, y, z, w, x, y, z) (pos + quat, scalar_first)."
        )

    # The robot_view expects a transformation matrix in the form of a 4x4 numpy array
    self.robot_view.get_move_group("base").pose = pose_tf

update_control

update_control(action_command_dict: dict[str, Any]) -> None

Update the control inputs to the robot based on the provided action commands. Args: action_command_dict: Dictionary containing action commands for the robot based on the move groups ids to be used.

Source code in molmo_spaces/robots/rby1.py

def update_control(self, action_command_dict: dict[str, Any]) -> None:
    """Update the control inputs to the robot based on the provided action commands.
    Args:
    action_command_dict: Dictionary containing action commands for the robot
                         based on the move groups ids to be used.
    """
    # Loops through all the controllers and updates their control inputs
    # NOTE: All joints are always controlled. If no action_command is provided, we assume the
    # controller will maintain the current state of the joint.

    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for move_group_id, controller in self.controllers.items():
        if move_group_id in action_command_dict:
            action_command = action_command_dict[move_group_id]
            # send command target for the controller
            controller.set_target(action_command)
        else:
            # If no action command is provided, controller should switch to
            # stationary mode (if not already set to stationary previously)
            if not controller.stationary:
                controller.set_to_stationary()

robot_views

Modules:

Name	Description
abstract	This module defines the core abstractions for representing and controlling robots in MuJoCo.
bimanual_yam_view	Implementation of the bimanual YAM robot model.
franka_cap_view
franka_droid_view
franka_fr3_view	Implementation of the Franka FR3 robot model.
i2rt_yam_view	Implementation of the i2rt YAM robot model.
rby1_view	Implementation of the RBY1 robot model.
rum_gripper_view
stretch_dex_view

abstract

This module defines the core abstractions for representing and controlling robots in MuJoCo. The architecture is based on a hierarchical structure where a RobotView contains multiple MoveGroups, each representing an atomic collection of joints and actuators.

The key abstractions are: - MoveGroup: Base class for any collection of joints and actuators - Arm: A MoveGroup with additional gripper functionality - RobotBase: A MoveGroup that controls the overall robot pose - RobotView: Top-level class that contains and manages multiple MoveGroups

Classes:

Name	Description
FreeJointRobotBaseGroup	A RobotBase that uses a free joint to represent its pose.
GripperGroup
HoloJointsRobotBaseGroup	A RobotBase that uses virtual holonomic joints to represent its pose.
ImmobileRobotBaseGroup	A RobotBase that is immobile and does not have a mocap body.
MocapRobotBaseGroup	A RobotBase that uses a mocap body to represent its pose.
MoveGroup	Base class for any collection of joints and actuators in a robot.
RobotBaseGroup	A MoveGroup that controls the overall pose of the robot.
RobotView	Top-level class that contains and manages multiple MoveGroups.

Attributes:

Name	Type	Description
RobotViewFactory	TypeAlias

RobotViewFactory module-attribute

RobotViewFactory: TypeAlias = Callable[[MjData, str], RobotView]

FreeJointRobotBaseGroup

FreeJointRobotBaseGroup(mj_data: MjData, base_joint_id: int, joint_ids: list[int], actuator_ids: list[int], floating=False)

Bases: RobotBaseGroup

A RobotBase that uses a free joint to represent its pose.

This implementation uses MuJoCo's free joint type to represent the base pose, which allows for full 6-DOF control of the robot's position and orientation.

Initialize a FreeJointRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
base_joint_id	int	The ID of the free joint that represents the base pose	required
joint_ids	list[int]	List of additional joint IDs that belong to this base	required
actuator_ids	list[int]	List of actuator IDs that control the base	required
floating		Whether the base is floating (concretely, whether IK should not constrain it to be on the ground)	False

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
floating		Whether the base is floating.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: MjData,
    base_joint_id: int,
    joint_ids: list[int],
    actuator_ids: list[int],
    floating=False,
):
    """Initialize a FreeJointRobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        base_joint_id: The ID of the free joint that represents the base pose
        joint_ids: List of additional joint IDs that belong to this base
        actuator_ids: List of actuator IDs that control the base
        floating: Whether the base is floating (concretely, whether IK should not constrain it to be on the ground)
    """
    base_body_id = mj_data.model.jnt_bodyid[base_joint_id]
    super().__init__(mj_data, [base_joint_id] + joint_ids, actuator_ids, base_body_id)
    assert self.mj_model.jnt_type[base_joint_id] == mujoco.mjtJoint.mjJNT_FREE
    self._base_joint_id = base_joint_id
    self._floating = floating

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

floating property

floating

Whether the base is floating.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl abstractmethod property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

GripperGroup

GripperGroup(mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int, robot_base_group: Optional[RobotBaseGroup] = None)

Bases: MoveGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	Optional[RobotBaseGroup]	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: MjData,
    joint_ids: list[int],
    actuator_ids: list[int],
    root_body_id: int,
    robot_base_group: Optional["RobotBaseGroup"] = None,
) -> None:
    """Initialize a MoveGroup.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        joint_ids: List of joint IDs that belong to this move group
        actuator_ids: List of actuator IDs that control the joints
        root_body_id: The ID of the root body of this move group
        robot_base_group: The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data
    self._joint_ids = joint_ids
    self._robot_base_group = robot_base_group
    self._root_body_id = root_body_id

    self._joint_posadr: list[int] = []
    self._joint_veladr: list[int] = []
    for i in joint_ids:
        n_pos_dim = 1
        n_vel_dim = 1
        if self.mj_model.jnt_type[i] == mujoco.mjtJoint.mjJNT_FREE:
            n_pos_dim = 7
            n_vel_dim = 6
        elif self.mj_model.jnt_type[i] == mujoco.mjtJoint.mjJNT_BALL:
            n_pos_dim = 4
            n_vel_dim = 3
        self._joint_posadr.extend(
            list(range(self.mj_model.jnt_qposadr[i], self.mj_model.jnt_qposadr[i] + n_pos_dim))
        )
        self._joint_veladr.extend(
            list(range(self.mj_model.jnt_dofadr[i], self.mj_model.jnt_dofadr[i] + n_vel_dim))
        )

    self._actuator_ids = actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist abstractmethod property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range abstractmethod property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world abstractmethod property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world abstractmethod property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian abstractmethod

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def get_jacobian(self) -> np.ndarray:
    """Returns the (6, model.nv) jacobian of the move group.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    raise NotImplementedError

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open abstractmethod

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def set_gripper_ctrl_open(self, open: bool) -> None:
    """Set the gripper commanded position to be fully open or closed.

    Args:
        open: True to open the gripper, False to close it
    """
    raise NotImplementedError

HoloJointsRobotBaseGroup

HoloJointsRobotBaseGroup(mj_data: MjData, world_site_id: int, holo_base_site_id: int, joint_ids: list[int], actuator_ids: list[int], root_body_id: int)

Bases: RobotBaseGroup

A RobotBase that uses virtual holonomic joints to represent its pose.

Assumes three virtual holonomic joints for x, y, and theta control.

Initialize a HoloJointsRobotBase that has virtual holonomic joints and uses site control.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
world_site_id	int	The ID of the world site	required
holo_base_site_id	int	The ID of the site that represents the holonomic base pose	required
joint_ids	list[int]	List of joint IDs that belong to the virtual holonomic base pose. NOTE: Assumed order is [x, y, theta].	required
actuator_ids	list[int]	List of actuator IDs that control the base	required

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the holonomic base actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: MjData,
    world_site_id: int,
    holo_base_site_id: int,
    joint_ids: list[int],
    actuator_ids: list[int],
    root_body_id: int,
):
    """Initialize a HoloJointsRobotBase that has virtual holonomic joints and uses site control.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        world_site_id: The ID of the world site
        holo_base_site_id: The ID of the site that represents the holonomic base pose
        joint_ids: List of joint IDs that belong to the virtual holonomic base pose.
                   NOTE: Assumed order is [x, y, theta].
        actuator_ids: List of actuator IDs that control the base
    """
    super().__init__(mj_data, joint_ids, actuator_ids, root_body_id)
    assert all(
        self.mj_model.jnt_type[jid] == mujoco.mjtJoint.mjJNT_HINGE
        or self.mj_model.jnt_type[jid] == mujoco.mjtJoint.mjJNT_SLIDE
        for jid in joint_ids
    ), "All holonomic joints must be position joints (hinge or slide)"
    self._world_site_id = world_site_id
    self._holo_base_site_id = holo_base_site_id

ctrl property writable

ctrl: ndarray

Current control signals for the holonomic base actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._holo_base_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

ImmobileRobotBaseGroup

ImmobileRobotBaseGroup(mj_data: MjData, robot_base_body_id: int)

Bases: RobotBaseGroup

A RobotBase that is immobile and does not have a mocap body.

This generally shouldn't be used, and MocapRobotBaseGroup should be preferred. If a scene is badly constructed and the robot base is not a mocap body, this might be necessary.

Initialize a ImmobileRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the simulation state	required
robot_base_body_id	int	The ID of the body that represents the robot base	required

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(self, mj_data: MjData, robot_base_body_id: int) -> None:
    """Initialize a ImmobileRobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the simulation state
        robot_base_body_id: The ID of the body that represents the robot base
    """
    super().__init__(mj_data, [], [], robot_base_body_id)
    self._robot_base_body_id = robot_base_body_id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

MocapRobotBaseGroup

MocapRobotBaseGroup(mj_data: MjData, robot_base_body_id: int)

Bases: RobotBaseGroup

A RobotBase that uses a mocap body to represent its pose.

Initialize a MocapRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the simulation state	required
robot_base_body_id	int	The ID of the mocap body that represents the robot base. Not the Mocap ID!	required

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(self, mj_data: MjData, robot_base_body_id: int):
    """Initialize a MocapRobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the simulation state
        robot_base_body_id: The ID of the mocap body that represents the robot base. Not the Mocap ID!
    """
    robot_base_mocap_id = mj_data.model.body_mocapid[robot_base_body_id]
    super().__init__(mj_data, [], [], robot_base_body_id)
    self._robot_base_mocap_id = robot_base_mocap_id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

MoveGroup

MoveGroup(mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int, robot_base_group: Optional[RobotBaseGroup] = None)

Bases: ABC

Base class for any collection of joints and actuators in a robot.

A MoveGroup represents an atomic collection of robotic joints and their associated actuators. It maintains a kinematic tree between a root frame and a leaf frame, and provides methods to control and query the state of its joints and actuators.

The class handles the low-level details of interfacing with MuJoCo's state vectors (qpos, qvel) and control signals (ctrl), providing a clean interface for higher-level control.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	Optional[RobotBaseGroup]	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: MjData,
    joint_ids: list[int],
    actuator_ids: list[int],
    root_body_id: int,
    robot_base_group: Optional["RobotBaseGroup"] = None,
) -> None:
    """Initialize a MoveGroup.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        joint_ids: List of joint IDs that belong to this move group
        actuator_ids: List of actuator IDs that control the joints
        root_body_id: The ID of the root body of this move group
        robot_base_group: The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data
    self._joint_ids = joint_ids
    self._robot_base_group = robot_base_group
    self._root_body_id = root_body_id

    self._joint_posadr: list[int] = []
    self._joint_veladr: list[int] = []
    for i in joint_ids:
        n_pos_dim = 1
        n_vel_dim = 1
        if self.mj_model.jnt_type[i] == mujoco.mjtJoint.mjJNT_FREE:
            n_pos_dim = 7
            n_vel_dim = 6
        elif self.mj_model.jnt_type[i] == mujoco.mjtJoint.mjJNT_BALL:
            n_pos_dim = 4
            n_vel_dim = 3
        self._joint_posadr.extend(
            list(range(self.mj_model.jnt_qposadr[i], self.mj_model.jnt_qposadr[i] + n_pos_dim))
        )
        self._joint_veladr.extend(
            list(range(self.mj_model.jnt_dofadr[i], self.mj_model.jnt_dofadr[i] + n_vel_dim))
        )

    self._actuator_ids = actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world abstractmethod property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl abstractmethod property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world abstractmethod property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian abstractmethod

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def get_jacobian(self) -> np.ndarray:
    """Returns the (6, model.nv) jacobian of the move group.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    raise NotImplementedError

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RobotBaseGroup

RobotBaseGroup(mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int)

Bases: MoveGroup

A MoveGroup that controls the overall pose of the robot.

A RobotBase represents the base of a robot, which can be either mobile (e.g., wheeled) or immobile (e.g., fixed to a table). It provides methods to control and query the overall pose of the robot in the world frame.

Initialize a RobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this base	required
actuator_ids	list[int]	List of actuator IDs that control the base	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self, mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int
):
    """Initialize a RobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        joint_ids: List of joint IDs that belong to this base
        actuator_ids: List of actuator IDs that control the base
    """
    super().__init__(mj_data, joint_ids, actuator_ids, root_body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl abstractmethod property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose abstractmethod property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian abstractmethod

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def get_jacobian(self) -> np.ndarray:
    """Returns the (6, model.nv) jacobian of the move group.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    raise NotImplementedError

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RobotView

RobotView(mj_data: MjData, move_groups: dict[str, MoveGroup])

Bases: ABC

Top-level class that contains and manages multiple MoveGroups.

A RobotView represents a complete robot, composed of multiple MoveGroups (arms, base, etc.). It provides a unified interface to control and query the state of all move groups, and handles the coordination between different parts of the robot.

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	RobotBaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(self, mj_data: MjData, move_groups: dict[str, MoveGroup]) -> None:
    """Initialize a RobotView.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        move_groups: Dictionary mapping move group names to their implementations
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data
    self._move_groups = move_groups

base abstractmethod property

base: RobotBaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name abstractmethod property

name: str

The name of this robot model.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

bimanual_yam_view

Implementation of the bimanual YAM robot model.

The bimanual YAM consists of two 6-DOF YAM arms with parallel grippers, positioned side by side (44cm apart), both facing forward.

Each arm component is implemented as a MoveGroup (left_arm, right_arm, left_gripper, right_gripper), with the overall robot structure managed by the BimanualYamRobotView class.

Classes:

Name	Description
BimanualYamArmGroup	6-DOF arm group for one side of the bimanual YAM robot.
BimanualYamBaseGroup	Base group for the bimanual YAM robot using mocap body control.
BimanualYamGripperGroup	Parallel gripper group for one side of the bimanual YAM robot.
BimanualYamRobotView	Robot view for the bimanual YAM (two 6-DOF arms with parallel grippers).

BimanualYamArmGroup

BimanualYamArmGroup(mj_data: MjData, side: Literal['left', 'right'], base_group: BimanualYamBaseGroup, namespace: str = '', grasp_site_name: str = 'grasp_site')

Bases: MoveGroup

6-DOF arm group for one side of the bimanual YAM robot.

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
side	str
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base_group: BimanualYamBaseGroup,
    namespace: str = "",
    grasp_site_name: str = "grasp_site",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    self._side = side
    # Prefix for this arm: namespace + side_
    self._arm_prefix = f"{namespace}{side}_"

    # 6 arm joints: joint1 through joint6
    joint_ids = [model.joint(f"{self._arm_prefix}joint{i + 1}").id for i in range(6)]
    # 6 arm actuators with the same names
    act_ids = [model.actuator(f"{self._arm_prefix}joint{i + 1}").id for i in range(6)]
    self._arm_root_id = model.body(f"{self._arm_prefix}arm").id
    self._ee_site_id = model.site(f"{self._arm_prefix}{grasp_site_name}").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base_group)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

side property

side: str

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

BimanualYamBaseGroup

BimanualYamBaseGroup(mj_data: MjData, namespace: str = '')

Bases: MocapRobotBaseGroup

Base group for the bimanual YAM robot using mocap body control.

The mocap body is created by add_robot_to_scene and serves as the root for both arms.

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    body_id: int = mj_data.model.body(f"{namespace}base").id
    super().__init__(mj_data, body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

BimanualYamGripperGroup

BimanualYamGripperGroup(mj_data: MjData, side: Literal['left', 'right'], base_group: BimanualYamBaseGroup, namespace: str = '')

Bases: GripperGroup

Parallel gripper group for one side of the bimanual YAM robot.

The YAM gripper has two coupled fingers (left_finger and right_finger) controlled by a single actuator. The fingers move in opposite directions due to an equality constraint.

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set gripper to fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	Distance between fingers.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
side	str
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base_group: BimanualYamBaseGroup,
    namespace: str = "",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    self._side = side
    # Prefix for this gripper: namespace + side_
    self._gripper_prefix = f"{namespace}{side}_"

    # Two coupled finger joints
    joint_ids = [
        model.joint(f"{self._gripper_prefix}left_finger").id,
        model.joint(f"{self._gripper_prefix}right_finger").id,
    ]
    # Single gripper actuator controls left_finger (right follows via equality constraint)
    act_ids = [model.actuator(f"{self._gripper_prefix}gripper").id]
    root_body_id = model.body(f"{self._gripper_prefix}link_6").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{self._gripper_prefix}grasp_site").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

Distance between fingers.

Since fingers are coupled in opposite directions (right = -left), the total opening is the sum of absolute positions.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers.

From XML: - left_finger range: -0.00205 to 0.037524 - right_finger range: -0.037524 to 0.00205 (mirrored) - Closed (both at 0): dist = 0 - Open (left at 0.037524, right at -0.037524): dist ~= 0.075

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

side property

side: str

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set gripper to fully open or closed.

From XML: gripper actuator ctrlrange is 0.0 to 0.041 - 0.0 = closed - 0.041 = open

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    """Set gripper to fully open or closed.

    From XML: gripper actuator ctrlrange is 0.0 to 0.041
    - 0.0 = closed
    - 0.041 = open
    """
    self.ctrl = [0.041 if open else 0.0]

BimanualYamRobotView

BimanualYamRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Robot view for the bimanual YAM (two 6-DOF arms with parallel grippers).

Move groups: - base: Mocap body controlling both arms - left_arm: Left 6-DOF arm - right_arm: Right 6-DOF arm - left_gripper: Left parallel gripper - right_gripper: Right parallel gripper

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	BimanualYamBaseGroup
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = BimanualYamBaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "left_arm": BimanualYamArmGroup(mj_data, "left", base, namespace=namespace),
        "right_arm": BimanualYamArmGroup(mj_data, "right", base, namespace=namespace),
        "left_gripper": BimanualYamGripperGroup(mj_data, "left", base, namespace=namespace),
        "right_gripper": BimanualYamGripperGroup(mj_data, "right", base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: BimanualYamBaseGroup

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

franka_cap_view

Classes:

Name	Description
CAPGripperGroup
FrankaCAPRobotView

CAPGripperGroup

CAPGripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: GripperGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float
inter_finger_dist_range	tuple[float, float]
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_cap_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}gripper/finger_left_joint").id,
        model.joint(f"{namespace}gripper/finger_right_joint").id,
    ]
    act_ids = [model.actuator(f"{namespace}gripper/fingers_actuator").id]
    root_body_id = model.body(f"{namespace}gripper/base").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}gripper/grasp_site").id
    self._finger_1_geom_id = model.geom(f"{namespace}gripper/left_pad2").id
    self._finger_2_geom_id = model.geom(f"{namespace}gripper/right_pad2").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/franka_cap_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Source code in molmo_spaces/robots/robot_views/franka_cap_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [0 if open else 255]

FrankaCAPRobotView

FrankaCAPRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FrankaFR3BaseGroup
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str

Source code in molmo_spaces/robots/robot_views/franka_cap_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FrankaFR3BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": FrankaFR3ArmGroup(
            mj_data, base, namespace=namespace, grasp_site_name="gripper/grasp_site"
        ),
        "gripper": CAPGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FrankaFR3BaseGroup

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

franka_droid_view

Classes:

Name	Description
FloatingRobotiq2f85BaseGroup
FloatingRobotiq2f85RobotView
FloatingRobotiqGripperGroup
FrankaDroidRobotView
RobotIQGripperGroup

FloatingRobotiq2f85BaseGroup

FloatingRobotiq2f85BaseGroup(mj_data: MjData, namespace: str = '')

Bases: FreeJointRobotBaseGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
is_mobile
n_actuators

Attributes:

Name	Type	Description
ctrl
ctrl_limits
floating		Whether the base is floating.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    model = mj_data.model
    base_joint_id = model.joint(f"{namespace}robotiq_2f85_free_joint").id
    self._target_pose_body = create_mlspaces_body(mj_data, f"{namespace}target_ee_pose")
    super().__init__(mj_data, base_joint_id, [], [], floating=True)

ctrl property writable

ctrl

ctrl_limits cached property

ctrl_limits

floating property

floating

Whether the base is floating.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

is_mobile

is_mobile()

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def is_mobile(self):
    return True

n_actuators

n_actuators()

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def n_actuators(self):
    return 7

FloatingRobotiq2f85RobotView

FloatingRobotiq2f85RobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FloatingRobotiq2f85BaseGroup
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FloatingRobotiq2f85BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "gripper": FloatingRobotiqGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FloatingRobotiq2f85BaseGroup

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

FloatingRobotiqGripperGroup

FloatingRobotiqGripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: RobotIQGripperGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float
inter_finger_dist_range	tuple[float, float]
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}left_driver_joint").id,
        model.joint(f"{namespace}right_driver_joint").id,
    ]
    act_ids = [model.actuator(f"{namespace}fingers_actuator").id]
    root_body_id = model.body(f"{namespace}base").id
    GripperGroup.__init__(self, mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id
    self._finger_1_geom_id = model.geom(f"{namespace}left_pad2").id
    self._finger_2_geom_id = model.geom(f"{namespace}right_pad2").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [0 if open else 255]

FrankaDroidRobotView

FrankaDroidRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FrankaFR3BaseGroup
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FrankaFR3BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": FrankaFR3ArmGroup(
            mj_data, base, namespace=namespace, grasp_site_name="gripper/grasp_site"
        ),
        "gripper": RobotIQGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FrankaFR3BaseGroup

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

RobotIQGripperGroup

RobotIQGripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: GripperGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float
inter_finger_dist_range	tuple[float, float]
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}gripper/left_driver_joint").id,
        model.joint(f"{namespace}gripper/right_driver_joint").id,
    ]
    act_ids = [model.actuator(f"{namespace}gripper/fingers_actuator").id]
    root_body_id = model.body(f"{namespace}gripper/base").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}gripper/grasp_site").id
    self._finger_1_geom_id = model.geom(f"{namespace}gripper/left_pad2").id
    self._finger_2_geom_id = model.geom(f"{namespace}gripper/right_pad2").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [0 if open else 255]

franka_fr3_view

Implementation of the Franka FR3 robot model.

The Franka FR3 is a single-arm 7-DOF robot with a gripper.

Each component is implemented as a MoveGroup, with the overall robot structure managed by the FrankaFR3RobotView class.

Classes:

Name	Description
FrankaFR3ArmGroup
FrankaFR3BaseGroup
FrankaFR3GripperGroup
FrankaFR3RobotView

FrankaFR3ArmGroup

FrankaFR3ArmGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '', grasp_site_name: str = 'grasp_site')

Bases: MoveGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(
    self,
    mj_data: MjData,
    base_group: FrankaFR3BaseGroup,
    namespace: str = "",
    grasp_site_name: str = "grasp_site",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [model.joint(f"{namespace}fr3_joint{i + 1}").id for i in range(7)]
    act_ids = [model.actuator(f"{namespace}fr3_joint{i + 1}").id for i in range(7)]
    self._arm_root_id = model.body(f"{namespace}fr3_link0").id
    self._ee_site_id = model.site(f"{namespace}{grasp_site_name}").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base_group)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

FrankaFR3BaseGroup

FrankaFR3BaseGroup(mj_data: MjData, namespace: str = '')

Bases: MocapRobotBaseGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    body_id: int = mj_data.model.body(f"{namespace}base").id
    super().__init__(mj_data, body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

FrankaFR3GripperGroup

FrankaFR3GripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: GripperGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float
inter_finger_dist_range	tuple[float, float]
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}finger_joint1").id,
        model.joint(f"{namespace}finger_joint2").id,
    ]
    act_ids = [model.actuator(f"{namespace}panda_hand").id]
    root_body_id = model.body(f"{namespace}hand").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [255 if open else 0]

FrankaFR3RobotView

FrankaFR3RobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FrankaFR3BaseGroup
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FrankaFR3BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": FrankaFR3ArmGroup(mj_data, base, namespace=namespace),
        "gripper": FrankaFR3GripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FrankaFR3BaseGroup

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

i2rt_yam_view

Implementation of the i2rt YAM robot model.

The YAM is a 6-DOF arm robot with a parallel gripper that has coupled fingers.

Each component is implemented as a MoveGroup, with the overall robot structure managed by the I2rtYamRobotView class.

Classes:

Name	Description
I2rtYamArmGroup	6-DOF arm group for the YAM robot.
I2rtYamBaseGroup	Base group for the YAM robot using mocap body control.
I2rtYamGripperGroup	Parallel gripper group for the YAM robot.
I2rtYamRobotView	Robot view for the i2rt YAM 6-DOF arm with parallel gripper.

I2rtYamArmGroup

I2rtYamArmGroup(mj_data: MjData, base_group: I2rtYamBaseGroup, namespace: str = '', grasp_site_name: str = 'grasp_site')

Bases: MoveGroup

6-DOF arm group for the YAM robot.

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(
    self,
    mj_data: MjData,
    base_group: I2rtYamBaseGroup,
    namespace: str = "",
    grasp_site_name: str = "grasp_site",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    # 6 arm joints: joint1 through joint6
    joint_ids = [model.joint(f"{namespace}joint{i + 1}").id for i in range(6)]
    # 6 arm actuators with the same names
    act_ids = [model.actuator(f"{namespace}joint{i + 1}").id for i in range(6)]
    self._arm_root_id = model.body(f"{namespace}arm").id
    self._ee_site_id = model.site(f"{namespace}{grasp_site_name}").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base_group)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

I2rtYamBaseGroup

I2rtYamBaseGroup(mj_data: MjData, namespace: str = '')

Bases: MocapRobotBaseGroup

Base group for the YAM robot using mocap body control.

Note: The mocap body is created by add_robot_to_scene and named 'base', not 'arm'. The 'arm' body from the XML is attached under this mocap body.

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    body_id: int = mj_data.model.body(f"{namespace}base").id
    super().__init__(mj_data, body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

I2rtYamGripperGroup

I2rtYamGripperGroup(mj_data: MjData, base_group: I2rtYamBaseGroup, namespace: str = '')

Bases: GripperGroup

Parallel gripper group for the YAM robot.

The YAM gripper has two coupled fingers (left_finger and right_finger) controlled by a single actuator. The fingers move in opposite directions due to an equality constraint: right_finger = -left_finger.

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set gripper to fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	Distance between fingers.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(self, mj_data: MjData, base_group: I2rtYamBaseGroup, namespace: str = "") -> None:
    model = mj_data.model
    self._namespace = namespace
    # Two coupled finger joints
    joint_ids = [
        model.joint(f"{namespace}left_finger").id,
        model.joint(f"{namespace}right_finger").id,
    ]
    # Single gripper actuator controls left_finger (right follows via equality constraint)
    act_ids = [model.actuator(f"{namespace}gripper").id]
    root_body_id = model.body(f"{namespace}link_6").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

Distance between fingers.

Since fingers are coupled in opposite directions (right = -left), the total opening is the sum of absolute positions.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers.

From XML: - left_finger range: -0.00205 to 0.037524 - right_finger range: -0.037524 to 0.00205 (mirrored) - Closed (both at 0): dist = 0 - Open (left at 0.037524, right at -0.037524): dist ~= 0.075

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set gripper to fully open or closed.

From XML: gripper actuator ctrlrange is 0.0 to 0.041 - 0.0 = closed - 0.041 = open

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    """Set gripper to fully open or closed.

    From XML: gripper actuator ctrlrange is 0.0 to 0.041
    - 0.0 = closed
    - 0.041 = open
    """
    self.ctrl = [0.041 if open else 0.0]

I2rtYamRobotView

I2rtYamRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Robot view for the i2rt YAM 6-DOF arm with parallel gripper.

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	I2rtYamBaseGroup
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = I2rtYamBaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": I2rtYamArmGroup(mj_data, base, namespace=namespace),
        "gripper": I2rtYamGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: I2rtYamBaseGroup

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

rby1_view

Implementation of the RBY1 robot model.

The RBY1 is a humanoid robot with: - Two 7-DOF arms with grippers - A 6-DOF torso - A mobile base with two wheels - A 2-DOF head

Each component is implemented as a MoveGroup, with the overall robot structure managed by the RBY1 RobotView class.

Classes:

Name	Description
RBY1ArmGroup	Implementation of the RBY1's 7-DOF arm, excluding the gripper.
RBY1BaseGroup	Implementation of the RBY1's mobile base.
RBY1GripperGroup	Implementation of the RBY1's gripper.
RBY1HeadGroup	Implementation of the RBY1's head.
RBY1HoloBaseGroup	Implementation of a RBY1 mobile base with virtual holonomic joints and site control.
RBY1RobotView	Implementation of the complete RBY1 robot.
RBY1TorsoGroup	Implementation of the RBY1's torso.

RBY1ArmGroup

RBY1ArmGroup(mj_data: MjData, side: Literal['left', 'right'], base: RobotBaseGroup, namespace: str = '')

Bases: MoveGroup

Implementation of the RBY1's 7-DOF arm, excluding the gripper.

Initialize an RBY1 arm.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
side	Literal['left', 'right']	Which side of the robot this arm is on ("left" or "right")	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
side
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base: RobotBaseGroup,
    namespace: str = "",
) -> None:
    """Initialize an RBY1 arm.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        side: Which side of the robot this arm is on ("left" or "right")
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    self.side = side
    self._namespace = namespace
    joint_ids = [model.joint(f"{namespace}{side}_arm_{i}").id for i in range(7)]
    act_ids = [model.actuator(f"{namespace}{side}_arm_{i + 1}_act").id for i in range(7)]
    self._ee_site_id = model.site(f"{namespace}ee_site_{side[0]}").id
    self._arm_root_id = model.body(f"{namespace}link_{side}_arm_0").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

side instance-attribute

side = side

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1BaseGroup

RBY1BaseGroup(mj_data: MjData, namespace: str = '')

Bases: FreeJointRobotBaseGroup

Implementation of the RBY1's mobile base.

The RBY1 base uses a free joint for its pose and has two wheels for mobility. The wheels are controlled independently, allowing for differential drive motion.

Initialize the RBY1 base.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
floating		Whether the base is floating.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    """Initialize the RBY1 base.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    base_joint_id = model.joint(f"{namespace}freejoint").id
    joints = [model.joint(f"{namespace}{side}_wheel").id for side in ["left", "right"]]
    act = [model.actuator(f"{namespace}{side}_wheel_act").id for side in ["left", "right"]]
    super().__init__(mj_data, base_joint_id, joints, act)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

floating property

floating

Whether the base is floating.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1GripperGroup

RBY1GripperGroup(mj_data: MjData, side: Literal['left', 'right'], base: RobotBaseGroup, namespace: str = '')

Bases: GripperGroup

Implementation of the RBY1's gripper.

The RBY1 gripper has 2 fingers that are mechanically coupled, allowing for open/close motion of the two fingers.

Initialize an RBY1 gripper.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
side	Literal['left', 'right']	Which side of the robot this gripper is on ("left" or "right")	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the gripper actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float
inter_finger_dist_range	tuple[float, float]
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base: RobotBaseGroup,
    namespace: str = "",
) -> None:
    """Initialize an RBY1 gripper.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        side: Which side of the robot this gripper is on ("left" or "right")
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    self._side = side
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}gripper_finger_{side[0]}{i + 1}").id for i in range(2)
    ]
    act_ids = [model.actuator(f"{namespace}{side}_finger_act").id]
    root_body_id = model.body(f"{namespace}EE_BODY_{side[0].upper()}")
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base)
    self._ee_site_id = model.site(f"{namespace}ee_site_{side[0]}").id

ctrl property writable

ctrl: ndarray

Current control signals for the gripper actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = np.array([-0.05 if open else 0.0])

RBY1HeadGroup

RBY1HeadGroup(mj_data: MjData, base: RobotBaseGroup, namespace: str = '')

Bases: MoveGroup

Implementation of the RBY1's head.

The RBY1 head has 2 degrees of freedom, allowing for pan and tilt motion to look around the environment.

Initialize the RBY1 head.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, base: RobotBaseGroup, namespace: str = "") -> None:
    """Initialize the RBY1 head.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    joint_ids = [model.joint(f"{namespace}head_{i}").id for i in range(2)]
    act_ids = [model.actuator(f"{namespace}head_{i}_act").id for i in range(2)]
    root_body_id = model.body(f"{namespace}link_head_1").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base)
    self._head_root_id = model.body(f"{namespace}link_head_1").id
    self._head_leaf_id = model.body(f"{namespace}link_head_2").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self._head_leaf_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1HoloBaseGroup

RBY1HoloBaseGroup(mj_data: MjData, namespace: str = '')

Bases: HoloJointsRobotBaseGroup

Implementation of a RBY1 mobile base with virtual holonomic joints and site control.

The RBY1 base uses three virtual holonomic joints for x, y and theta control.

Initialize the RBY1 holo base.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the holonomic base actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    """Initialize the RBY1 holo base.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    world_site_id = model.site(f"{namespace}world").id
    holo_base_site_id = model.site(f"{namespace}base_site").id
    joints = [model.joint(f"{namespace}base_{axis}").id for axis in ["x", "y", "theta"]]
    act = [model.actuator(f"{namespace}base_{axis}_act").id for axis in ["x", "y", "theta"]]
    root_body_id = model.body(f"{namespace}base")
    super().__init__(mj_data, world_site_id, holo_base_site_id, joints, act, root_body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the holonomic base actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._holo_base_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1RobotView

RBY1RobotView(mj_data: MjData, namespace: str = '', holo_base: bool = False)

Bases: RobotView

Implementation of the complete RBY1 robot.

The RBY1 is a humanoid robot with: - Two 7-DOF arms with grippers - A 6-DOF torso - A mobile base with two wheels (or three holonomic joints) - A 2-DOF head

Each component is implemented as a MoveGroup, with the overall robot structure managed by this class.

Initialize the RBY1 robot.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
distance_to	Calculate the distance between the current joint positions of the move groups and the target pose
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_joint_position	Get the current joint positions of the move groups
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
is_close_to	Check if the current joint positions of the move groups are close to the target pose
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, namespace: str = "", holo_base: bool = False) -> None:
    """Initialize the RBY1 robot.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    self._namespace = namespace
    base = (
        RBY1BaseGroup(mj_data, namespace=namespace)
        if not holo_base
        else RBY1HoloBaseGroup(mj_data, namespace=namespace)
    )
    move_groups = {
        "base": base,
        "torso": RBY1TorsoGroup(mj_data, base, namespace=namespace),
        "left_arm": RBY1ArmGroup(mj_data, "left", base, namespace=namespace),
        "right_arm": RBY1ArmGroup(mj_data, "right", base, namespace=namespace),
        "left_gripper": RBY1GripperGroup(mj_data, "left", base, namespace=namespace),
        "right_gripper": RBY1GripperGroup(mj_data, "right", base, namespace=namespace),
        "head": RBY1HeadGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(
        mj_data,
        move_groups,
    )

base property

base

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

distance_to

distance_to(move_group_ids: list[str], target_pose: list) -> float

Calculate the distance between the current joint positions of the move groups and the target pose

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def distance_to(self, move_group_ids: list[str], target_pose: list) -> float:
    """Calculate the distance between the current joint positions of the move groups and the target pose"""
    assert len(target_pose) == 3, f"Expected [x, y, theta] pose, got {target_pose}"
    current_joint_pos = self.get_joint_position(move_group_ids)
    x_delta = current_joint_pos[0] - target_pose[0]
    y_delta = current_joint_pos[1] - target_pose[1]
    theta_delta = normalize_ang_error(current_joint_pos[2] - target_pose[2])
    return float(np.linalg.norm(np.array([x_delta, y_delta, theta_delta])))

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_joint_position

get_joint_position(move_group_ids: list[str]) -> ndarray

Get the current joint positions of the move groups

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def get_joint_position(self, move_group_ids: list[str]) -> np.ndarray:
    """Get the current joint positions of the move groups"""
    return np.concatenate(
        [
            self.get_move_group(move_group_id).joint_pos.copy()
            for move_group_id in move_group_ids
        ]
    )

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

is_close_to

is_close_to(move_group_ids: list[str], target_pose: list, threshold: float = 0.05) -> bool

Check if the current joint positions of the move groups are close to the target pose

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def is_close_to(
    self, move_group_ids: list[str], target_pose: list, threshold: float = 0.05
) -> bool:
    """Check if the current joint positions of the move groups are close to the target pose"""
    return self.distance_to(move_group_ids, target_pose) < threshold

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

RBY1TorsoGroup

RBY1TorsoGroup(mj_data: MjData, base: RobotBaseGroup, namespace: str = '')

Bases: MoveGroup

Implementation of the RBY1's torso.

The RBY1 torso has 6 degrees of freedom, allowing for full control of the upper body's position and orientation relative to the base.

Initialize the RBY1 torso.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, base: RobotBaseGroup, namespace: str = "") -> None:
    """Initialize the RBY1 torso.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [model.joint(f"{namespace}torso_{i}").id for i in range(6)]
    act_ids = [model.actuator(f"{namespace}link{i + 1}_act").id for i in range(6)]
    self._torso_root_id = model.body(f"{namespace}link_torso_0").id
    self._torso_leaf_id = model.body(f"{namespace}link_torso_5").id
    super().__init__(mj_data, joint_ids, act_ids, self._torso_root_id, base)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self._torso_leaf_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

rum_gripper_view

Classes:

Name	Description
FloatingRUMBaseGroup	A robot base group for the floating RUM gripper.
FloatingRUMRobotView
RUMGripperGroup

Functions:

Name	Description
main

FloatingRUMBaseGroup

FloatingRUMBaseGroup(mj_data: MjData, namespace: str = '')

Bases: FreeJointRobotBaseGroup

A robot base group for the floating RUM gripper. The gripper is controlled via a target pose mocap body. Since this doesn't correspond to an actuator, we "fake" the actuators by overriding the appropriate methods and properties.

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl
ctrl_limits
floating		Whether the base is floating.
is_mobile
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators
n_joints	int	Number of joints in this move group.
noop_ctrl
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def __init__(self, mj_data: MjData, namespace: str = ""):
    model = mj_data.model
    base_joint_id = model.joint(f"{namespace}rum_free_joint").id
    self._target_pose_body = create_mlspaces_body(mj_data, f"{namespace}target_ee_pose")
    super().__init__(mj_data, base_joint_id, [], [], floating=True)

ctrl property writable

ctrl

ctrl_limits cached property

ctrl_limits

floating property

floating

Whether the base is floating.

is_mobile cached property

is_mobile

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

FloatingRUMRobotView

FloatingRUMRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FloatingRUMBaseGroup
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def __init__(self, mj_data: MjData, namespace: str = ""):
    self._namespace = namespace
    base = FloatingRUMBaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "gripper": RUMGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FloatingRUMBaseGroup

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

RUMGripperGroup

RUMGripperGroup(mj_data: MjData, base_group: RobotBaseGroup, namespace: str = '')

Bases: GripperGroup

Methods:

Name	Description
get_jacobian
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float
inter_finger_dist_range	tuple[float, float]
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
noop_ctrl	ndarray
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def __init__(self, mj_data: MjData, base_group: RobotBaseGroup, namespace: str = ""):
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}finger_left_joint").id,
        model.joint(f"{namespace}finger_right_joint").id,
    ]
    actuator_ids = [model.actuator(f"{namespace}fingers_actuator").id]
    super().__init__(mj_data, joint_ids, actuator_ids, base_group.root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id
    self._left_fingertip_geom_id = model.geom(f"{namespace}left_fingertip").id
    self._right_fingertip_geom_id = model.geom(f"{namespace}right_fingertip").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

noop_ctrl property

noop_ctrl: ndarray

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mujoco.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._ee_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns: Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate
    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool)

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def set_gripper_ctrl_open(self, open: bool):
    self.ctrl = [0 if open else -255]

main

main()

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def main():
    import time

    import mujoco
    import numpy as np
    from mujoco import MjData, MjModel
    from mujoco.viewer import launch_passive

    np.set_printoptions(linewidth=np.inf)

    xml_path = str(get_robot_paths().get("floating_rum")) + "/model.xml"
    model = MjModel.from_xml_path(xml_path)

    data = MjData(model)
    mujoco.mj_forward(model, data)

    ns = ""
    robot_view = FloatingRUMRobotView(data, ns)
    robot_view.base.pose = np.eye(4)
    robot_view.base.ctrl = np.array([0, 0, 0, 1.0, 0, 0, 0])

    with launch_passive(model, data) as viewer:
        while viewer.is_running():
            viewer.sync()
            x = np.sin(data.time)
            y = 1 - np.cos(data.time)
            robot_view.base.ctrl = np.array([x, y, 0, 1.0, 0, 0, 0])
            mujoco.mj_step(model, data)
            time.sleep(0.02)

stretch_dex_view

Classes:

Name	Description
NamespaceDictWrapper
StretchDexRobotView

NamespaceDictWrapper

NamespaceDictWrapper(namespace: str, d: dict[str, Any])

Methods:

Name	Description
__contains__
__getitem__
__len__
__setitem__
keys

Attributes:

Name	Type	Description
d
namespace

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def __init__(self, namespace: str, d: dict[str, Any]) -> None:
    assert isinstance(d, Mapping)
    self.namespace = namespace
    self.d = d

d instance-attribute

d = d

namespace instance-attribute

namespace = namespace

__contains__

__contains__(item: str) -> bool

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def __contains__(self, item: str) -> bool:
    return self.namespace + item in self.d

__getitem__

__getitem__(item: str)

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def __getitem__(self, item: str):
    return self.d[self.namespace + item]

__len__

__len__() -> int

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def __len__(self) -> int:
    return len(self.d)

__setitem__

__setitem__(key: str, value: Any) -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def __setitem__(self, key: str, value: Any) -> None:
    self.d[self.namespace + key] = value

keys

keys()

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def keys(self):
    return [k[len(self.namespace) :] for k in self.d]

StretchDexRobotView

StretchDexRobotView(model: MjModel, namespace: str = 'robot_0/')

Bases: RobotView

Methods:

Name	Description
__call__
calculate_head_angles_to_look_at_object	Calculate the head angles to look at an object
check_collision
check_target_in_view
ee_jacobian
from_model
get_actuator_joints
get_camera_pose_rel_base
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_object_picked_up
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
init_gripper_pos
init_joint_pos
move_group_ids	Get the IDs of all move groups in this robot.
reset
set_joint_qpos
set_qpos_dict	Set the joint positions of all move groups.
update
update_actuator_ctrl_inputs
update_camera_poses
update_ee_jacobian
update_joints_pos
update_rotation_matrix

Attributes:

Name	Type	Description
GRIPPER_LEFT_NAME
GRIPPER_RIGHT_NAME		Default rotation matrix for the wrist when in top-down grasping pose.
INIT_JOINT_POS
STRETCH_TOPDOWN_WRIST_ROTATION		Default joint positions for top-down grasping pose.
TOPDOWN_JOINT_POS
actuator_ctrl_inputs
actuator_to_joints
all_joint_pos		all joints arm, gripper, head
angvel
arm_joint_pos
base	RobotBaseGroup	The base of the robot.
base_pose
camera_names
ee_pose_from_base
finger_body_ids
grasp_center_from_base
gripper_vel	NoReturn
has_fallen
head_camera_pose
in_view_camera_name	str
joint_limits
joint_pos		for planner. corresponing to state dim
joint_pos_names		in the order ctrl but index meant for joint_pos property
joint_pos_names_all		in the order ctrl but index meant for joint_pos property
joints
linvel
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str
namespace
position
quaternion
root_id	int
rotation_matrix
state_space_dim
wrist_camera_pose

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def __init__(self, model: MjModel, namespace: str = "robot_0/") -> None:
    super().__init__(model)
    self._state_space_dim = 8  # 3 base planar joint + 5 arm joints used for planner
    self.namespace = namespace

    base_id = model.body(self.namespace).id  # + "base_link").id
    self._root_id = model.body_rootid[base_id]

    self._get_gripper_geom_names()
    self._set_joint_names(model)
    self._set_arm_joint_limits(model)
    self._set_actuator_to_joints(model)
    self.reset()

GRIPPER_LEFT_NAME class-attribute instance-attribute

GRIPPER_LEFT_NAME = 'rubber_tip_left'

GRIPPER_RIGHT_NAME class-attribute instance-attribute

GRIPPER_RIGHT_NAME = 'rubber_tip_right'

Default rotation matrix for the wrist when in top-down grasping pose. This 3x3 matrix represents the orientation of the end-effector for a Stretch3 with DexWrist in top-down pose. The wrist is aligned vertically above the target.

INIT_JOINT_POS class-attribute instance-attribute

INIT_JOINT_POS = array([0.0, 0.0, -1.57, 0.1, 0.0, 0.0, -1.57, 0.0, 0.04, 0.0, 0.0])

STRETCH_TOPDOWN_WRIST_ROTATION class-attribute instance-attribute

STRETCH_TOPDOWN_WRIST_ROTATION = array([[1.0, 0.02266, 0.0], [0.02256, -0.996, -0.0861], [0.0021403, 0.0860793, -1.0]])

Default joint positions for top-down grasping pose. 8-dimensional array corresponding to: [lift, arm_extend, wrist_yaw, wrist_pitch, wrist_roll, gripper, head_pan, head_tilt] The -1.57 (~90 degrees) pitch angle orients the gripper downward. The 0.1 lift position lifts the gripper up so the gripper does not touch the base.

TOPDOWN_JOINT_POS class-attribute instance-attribute

TOPDOWN_JOINT_POS = array([0.1, 0.0, 0.0, -1.57, 0.0, 0.04, 0.0, 0.0])

actuator_ctrl_inputs property

actuator_ctrl_inputs

actuator_to_joints property

actuator_to_joints

all_joint_pos property

all_joint_pos

all joints arm, gripper, head

angvel property

angvel

arm_joint_pos property

arm_joint_pos

base abstractmethod property

base: RobotBaseGroup

The base of the robot.

base_pose property

base_pose

camera_names property

camera_names

ee_pose_from_base property

ee_pose_from_base

finger_body_ids property

finger_body_ids

grasp_center_from_base property

grasp_center_from_base

gripper_vel property

gripper_vel: NoReturn

has_fallen property

has_fallen

head_camera_pose property

head_camera_pose

in_view_camera_name property

in_view_camera_name: str

joint_limits property

joint_limits

joint_pos property

joint_pos

for planner. corresponing to state dim

joint_pos_names property

joint_pos_names

in the order ctrl but index meant for joint_pos property

joint_pos_names_all property

joint_pos_names_all

in the order ctrl but index meant for joint_pos property

joints property

joints

linvel property

linvel

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

namespace instance-attribute

namespace = namespace

position property

position

quaternion property

quaternion

root_id property

root_id: int

rotation_matrix property

rotation_matrix

state_space_dim property writable

state_space_dim

wrist_camera_pose property

wrist_camera_pose

__call__

__call__(model: MjModel, data: MjData)

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def __call__(self, model: MjModel, data: MjData):
    assert self.model is model, "[StretchDexRobotView.__call__] Model mismatch"
    self.update(data)
    return self

calculate_head_angles_to_look_at_object

calculate_head_angles_to_look_at_object(object_transform, frame='global', base_transform=None, camera_transform=None, model=None, data=None)

Calculate the head angles to look at an object

Parameters:

Name	Type	Description	Default
base_transform	_type_	description	None
camera_transform	_type_	description	None
object_transform	_type_	description	required
frame	str	description. Defaults to "global".	'global'

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def calculate_head_angles_to_look_at_object(
    self,
    object_transform,
    frame="global",
    base_transform=None,
    camera_transform=None,
    model=None,
    data=None,
):
    """
    Calculate the head angles to look at an object

    Args:
        base_transform (_type_): _description_
        camera_transform (_type_): _description_
        object_transform (_type_): _description_
        frame (str, optional): _description_. Defaults to "global".
    """
    if base_transform is None:
        assert model is not None and data is not None
        base_transform = self.__call__(model, data).base_pose
    if camera_transform is None:
        assert model is not None and data is not None
        camera_transform = self.__call__(model, data).head_camera_pose

    camera_transform_from_base = global_to_relative_transform(camera_transform, base_transform)
    object_transform_from_base = global_to_relative_transform(object_transform, base_transform)

    look_at_vector = object_transform_from_base[:3, 3] - camera_transform_from_base[:3, 3]
    look_at_vector = look_at_vector / np.linalg.norm(look_at_vector)

    head_pan = np.arctan2(look_at_vector[1], look_at_vector[0])
    head_pan = np.mod(head_pan + np.pi, 2 * np.pi) - np.pi  # wrap to [-pi, pi]
    head_tilt = np.arctan2(look_at_vector[2], np.linalg.norm(look_at_vector[:2]))

    return head_pan, head_tilt

check_collision

check_collision(model: MjModel, data: MjData, grasped_objs: set[int] = None) -> bool

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def check_collision(self, model: MjModel, data: MjData, grasped_objs: set[int] = None) -> bool:
    assert self.model == model
    # TODO: ignore finger collision with grasped objects
    # check if the agent is in collision with any geoms in the scene
    agent_id = self.root_id
    contacts = data.contact
    for c in contacts:
        if c.exclude != 0:
            continue
        body1 = self.model.geom_bodyid[c.geom1]
        body2 = self.model.geom_bodyid[c.geom2]

        rootbody1 = self.model.body_rootid[body1]
        rootbody2 = self.model.body_rootid[body2]

        if rootbody1 == agent_id or rootbody2 == agent_id:
            # Ignore Collision with gripper tips and target object
            if (
                self.model.body(body1).name == self.left_gripper_geom_name
                or self.model.body(body1).name == self.right_gripper_geom_name
            ) or (
                self.model.body(body2).name == self.left_gripper_geom_name
                or self.model.body(body2).name == self.right_gripper_geom_name
            ):
                if grasped_objs is not None and grasped_objs:
                    if (
                        self.model.body(body1).id in grasped_objs
                        or self.model.body(body2).id in grasped_objs
                    ):
                        continue
                else:
                    continue

            # Ignore Collision with itself
            if rootbody1 == rootbody2 and rootbody1 == agent_id:
                continue

            # Collision with the floor
            if (
                self.model.body(rootbody2).name == "floor"
                or self.model.body(rootbody1).name == "floor"
                or self.model.body(rootbody2).name == "world"
                or self.model.body(rootbody1).name == "world"
            ):
                if (
                    self.model.body(rootbody2).name == self.namespace + "link_left_wheel"
                    or self.model.body(rootbody2).name == self.namespace + "link_right_wheel"
                    or self.model.body(rootbody1).name == self.namespace + "link_left_wheel"
                    or self.model.body(rootbody1).name == self.namespace + "link_right_wheel"
                    or self.model.body(rootbody1).name == self.namespace + "base_link"
                    or self.model.body(rootbody2).name == self.namespace + "base_link"
                    or self.model.body(rootbody1).name == self.namespace
                    or self.model.body(rootbody2).name == self.namespace
                ):
                    # print("Collision with floor detected", model.body(body1).name, model.body(body2).name)
                    continue
                # print(
                #    "Collision with floor detected",
                #    self.model.body(body1).name,
                #    self.model.body(body2).name,
                # )
                # return False
            print(
                "Collision detected",
                self.model.body(body1).name,
                self.model.body(body2).name,
            )
            return True
    return False

check_target_in_view

check_target_in_view(target_id: int, data: MjData, camera_name: str, visualize: bool = False, device_id: int = None)

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def check_target_in_view(
    self,
    target_id: int,
    data: MjData,
    camera_name: str,
    visualize: bool = False,
    device_id: int = None,
):
    # NOTE: This  will help double check if the target is in view
    # target_id = self.model.body(target_id).rootid.item()

    # assert self.model.camera()
    if not hasattr(self, "_renderer") or self._renderer is None:
        from molmo_spaces.renderer.opengl_rendering import MjOpenGLRenderer

        self._renderer = MjOpenGLRenderer(model=self.model, device_id=device_id)

    self._renderer.enable_segmentation_rendering()
    self._renderer.update(data, camera=camera_name)
    seg = self._renderer.render()[..., 2]  # (object ID, object type, body ID)

    # get all children bodyids of the target
    # Check if target is in view
    target_seg = np.isin(
        seg,
        [i for i in range(self.model.nbody) if self.model.body(i).rootid.item() == target_id],
    )

    if visualize:
        self._renderer.disable_segmentation_rendering()
        self._renderer.update(data, camera=camera_name)
        img = self._renderer.render()

        # blend original image with target segment
        overlay = img.copy()
        overlay[target_seg] = [0, 255, 0]  # green
        alpha = 0.5
        highlighted_img = cv2.addWeighted(img, 1 - alpha, overlay, alpha, 0)
        highlighted_img = cv2.cvtColor(highlighted_img, cv2.COLOR_BGR2RGB)

        # self.model.body(target_id).name
        plt.imshow(highlighted_img)
        plt.show()
        # cv2.imshow(f"{target_name} in view", highlighted_img)
        # cv2.waitKey(1)
        # cv2.destroyAllWindows()

    return target_seg.any()

ee_jacobian

ee_jacobian(model: MjModel, data: MjData)

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def ee_jacobian(self, model: MjModel, data: MjData):
    self.update_ee_jacobian(model, data)
    return self._ee_jacobian

from_model classmethod

from_model(model, namespace: str = 'robot_0/')

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

@classmethod
def from_model(cls, model, namespace: str = "robot_0/"):
    return StretchDexRobotView(model, namespace)

get_actuator_joints

get_actuator_joints(actuator_names)

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def get_actuator_joints(self, actuator_names):
    _actuator_joints = np.zeros(len(actuator_names))
    for i, act_name in enumerate(actuator_names):
        joint_names = self.actuator_to_joints.d[act_name]
        joint_index = [self.joint_names.index(j) for j in joint_names]
        _actuator_joints[i] = np.sum(self._joints[joint_index])
    return _actuator_joints

get_camera_pose_rel_base

get_camera_pose_rel_base(camera_name: str, model: MjModel, data: MjData)

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def get_camera_pose_rel_base(self, camera_name: str, model: MjModel, data: MjData):
    assert self.model == model
    self.update(data)
    camera = self.model.camera(self.namespace + camera_name)
    camera_body_id = camera.bodyid[0]

    # Get world position and orientation of camera body from data
    pos = data.xpos[camera_body_id].copy()
    rot = data.xmat[camera_body_id].reshape(3, 3).copy()

    # Combine into transformation matrix
    T_world = np.eye(4)
    T_world[:3, :3] = rot
    T_world[:3, 3] = pos
    return global_to_relative_transform(T_world, self.base_pose)

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns: The (6, N) jacobian of the move group, where N is the total number of degrees of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input
    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mujoco.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_object_picked_up

get_object_picked_up(model: MjModelBindings, data: MjData)

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def get_object_picked_up(self, model: MjModelBindings, data: MjData):
    assert self.model == model.model
    # check if the agent is touching any object
    # check if the object is picked up.
    # return that object id or name
    left_id = self.model.geom(self.left_gripper_geom_name).id
    right_id = self.model.geom(self.right_gripper_geom_name).id
    finger_tip_id = [left_id, right_id]

    gripper_object = []
    contacts = data.contact
    for c in contacts:
        if c.geom1 in finger_tip_id and c.geom2 in finger_tip_id:
            continue
        elif c.geom1 in finger_tip_id:
            body_id = self.model.geom_bodyid[c.geom2]
            body_name = self.model.body(body_id).name
            if body_name not in gripper_object:
                gripper_object.append(body_name)
        elif c.geom2 in finger_tip_id:
            body_id = self.model.geom_bodyid[c.geom1]
            body_name = self.model.body(body_id).name
            if body_name not in gripper_object:
                gripper_object.append(body_name)
    return gripper_object

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns: A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.
    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

init_gripper_pos

init_gripper_pos() -> NoReturn

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def init_gripper_pos() -> NoReturn:
    raise NotImplementedError

init_joint_pos

init_joint_pos()

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def init_joint_pos():
    return StretchDexRobotView.TOPDOWN_JOINT_POS.copy()

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

reset

reset() -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def reset(self) -> None:
    # base
    self._position = None
    self._quaternion = None
    self._rotation_matrix = None
    self._linvel = None
    self._angvel = None
    # arm joints
    self._joints = None
    # gripper
    self._grasp_center_position = None
    self._grasp_center_quaternion = None
    # actuator
    self._actuator_ctrl_inputs = None
    # planner base joint [x,y,theta]
    self._start_base_pose = None

set_joint_qpos

set_joint_qpos(model: MjModel, data: MjData, joint_pos: ndarray) -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def set_joint_qpos(self, model: MjModel, data: MjData, joint_pos: np.ndarray) -> None:
    assert self.model is model, "[StretchDexRobotView.set_joint_qpos] Model mismatch"
    for actuator, joints in self.actuator_to_joints.d.items():
        if actuator not in self.joint_pos_names_all:
            continue
        n = len(joints)
        for j in joints:
            bodyid = self.model.joint(self.namespace + j).bodyid.item()
            qposadr = self.model.joint(self.namespace + j).qposadr.item()
            dim = self.model.body(bodyid).dofnum.item()
            index = self.joint_pos_names_all.index(actuator)
            joint_target = joint_pos[index] / float(n)
            data.qpos[qposadr : qposadr + dim] = joint_target
        data.ctrl[2:] = joint_pos  # set actuator control inputs

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

update

update(data: MjData) -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def update(self, data: MjData) -> None:
    self._position = self._read_from_sensor("base_position", data)
    self._quaternion = self._read_from_sensor("base_quaternion", data)
    self._linvel = self._read_from_sensor("base_linvel", data)
    self._angvel = self._read_from_sensor("base_angvel", data)
    self._grasp_center_position = self._read_from_sensor("grasp_center_pos_from_base", data)
    self._grasp_center_quaternion = self._read_from_sensor("grasp_center_quat_from_base", data)
    self.update_actuator_ctrl_inputs(data)
    self.update_joints_pos(data)
    self.update_rotation_matrix()
    self.update_camera_poses(data)

update_actuator_ctrl_inputs

update_actuator_ctrl_inputs(data) -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def update_actuator_ctrl_inputs(self, data) -> None:
    n = self.model.nu
    ctrl_inputs = np.zeros(n)
    for i in range(n):
        if self.model.actuator(i).name.startswith(self.namespace):
            ctrl_inputs[i] = data.actuator(self.model.actuator(i).name).ctrl[0]
    self._actuator_ctrl_inputs = ctrl_inputs

update_camera_poses

update_camera_poses(data) -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def update_camera_poses(self, data) -> None:
    def pose_from_data(data, body_id):
        pose = np.eye(4)
        pose[:3, :3] = data.xmat[body_id].reshape(3, 3)
        pose[:3, 3] = data.xpos[body_id]
        return pose

    head_camera = self.model.body(self.namespace + "realsense").id
    self._head_camera_pose = pose_from_data(data, head_camera)
    wrist_camera = self.model.body(self.namespace + "d405_cam").id
    self._wrist_camera_pose = pose_from_data(data, wrist_camera)

update_ee_jacobian

update_ee_jacobian(model: MjModel, data: MjData) -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def update_ee_jacobian(self, model: MjModel, data: MjData) -> None:
    assert self.model is model, "[StretchDexRobotView.update_ee_jacobian] Model mismatch"
    # Jacobian for all sites
    site_names = [
        "base",
        "lift",
        "link_arm_l0",
        "link_arm_l1",
        "link_arm_l2",
        "link_arm_l3",
        "wrist_yaw",
        "wrist_pitch",
        "wrist_roll",
    ]  # , "link_grasp_center"]
    arm_extend_sites = ["link_arm_l0", "link_arm_l1", "link_arm_l2", "link_arm_l3"]

    # Compute Jacobian for each site
    n_sites = len(site_names)
    J_sites = np.empty((6 * n_sites, self.model.nv))
    for i, site_name in enumerate(site_names):
        site_id = self.model.site(self.namespace + site_name).id
        jacp = J_sites[6 * i : 6 * i + 3]
        jacr = J_sites[6 * i + 3 : 6 * i + 6]
        mujoco.mj_jacSite(self.model, data, jacp, jacr, site_id)
        # void mj_jacSite(const mjModel* m, const mjData* d, mjtNum* jacp, mjtNum* jacr, int site) {
        #    mj_jac(m, d, jacp, jacr, d->site_xpos + 3*site, m->site_bodyid[site]);
        #    }

    # Final jacobian
    J = np.zeros((6, self.state_space_dim))  # grasp center
    offset = 0
    if self.state_space_dim == 8:
        # Map base jacobian - world frame
        # 6 dof (xyz linear. xyz angular)
        base_dof_adr = self.model.body(self.root_id).dofadr[0]
        # but index 0,1,2 is in robot frame...
        J[:, 0] = J_sites[:6, base_dof_adr + 0]  # X Translation
        J[:, 1] = J_sites[:6, base_dof_adr + 1]  # Y Translation

        J[:, 2] = J_sites[:6, base_dof_adr + 5]  # Theta Rotation
        offset = 3

    # Map arm joints jacobian
    for _i, site_name in enumerate(site_names[1:]):
        if site_name in arm_extend_sites:
            if site_name != "link_arm_l0":
                continue  # already
            for arm_site in arm_extend_sites:
                # For primsatic arm joint, combine all four joint jacobians
                index = site_names.index(arm_site)
                site_body_id = self.model.site(self.namespace + site_name).bodyid[0]
                joint_id = self.model.body(site_body_id).jntadr[0]
                joint_dofadr = self.model.jnt(joint_id).dofadr[0]
                J[:, offset] += J_sites[6 * (index) : 6 * (index + 1), joint_dofadr]
        else:
            index = site_names.index(site_name)
            site_body_id = self.model.site(self.namespace + site_name).bodyid[0]
            joint_id = self.model.body(site_body_id).jntadr[0]
            joint_dofadr = self.model.jnt(joint_id).dofadr[0]
            J[:, offset] = J_sites[6 * (index) : 6 * (index + 1), joint_dofadr]
        offset += 1
    # 6 rows for spatial velocity [dx,dy, dz, wx, wy, wz]
    # self.state_space_dim columns for either planar base [0:2] and/or just arm joints [3:7]
    self._ee_jacobian = J

update_joints_pos

update_joints_pos(data: MjData) -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def update_joints_pos(self, data: MjData) -> None:
    _joints = np.zeros(len(self.joint_names))
    for i, joint_name in enumerate(self.joint_names):
        _joints[i] = self._read_from_sensor(joint_name, data).item()
    self._joints = _joints

update_rotation_matrix

update_rotation_matrix() -> None

Source code in molmo_spaces/robots/robot_views/stretch_dex_view.py

def update_rotation_matrix(self) -> None:
    #  update_base_transform(self, data: MjData):
    r = R.from_quat(self.quaternion, scalar_first=True).as_matrix()
    t = self.position
    transform = np.eye(4)
    transform[0:3, 0:3] = r
    transform[0:3, 3] = t
    self._rotation_matrix = transform