robots

robots

Modules:

Name	Description
abstract
bimanual_yam
find_gripper_finger_range	Script to find the inter finger distance range for all gripper move groups in a robot.
floating_robotiq
floating_rum
franka
g1	The G1 humanoid robot: g1_molmo's components/robot.py on molmo_spaces'
i2rt_yam
mobile_franka
rby1
robot_views
utils

abstract

Classes:

Name	Description
Robot

Attributes:

Name	Type	Description
log

log module-attribute

log = logging.getLogger(__name__)

Robot

Robot(mj_data: MjData, robot_config: BaseRobotConfig)

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers	dict[str, Controller]	One or more controllers for the robot joints / wheels / etc.
kinematics	MlSpacesKinematics	kinematic solver for the robot
mj_data
mj_model
namespace	str	robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics	ParallelKinematics	parallel kinematic solver for the robot
robot_config
robot_view	RobotView	robot view for interfacing with joints / links / actuators

Source code in molmo_spaces/robots/abstract.py

def __init__(self, mj_data: MjData, robot_config: BaseRobotConfig) -> None:
    self.mj_model = mj_data.model
    self.mj_data = mj_data
    self.robot_config = robot_config

    self._last_unnoised_cmd_joint_pos: dict[str, np.ndarray] | None = None

controllers abstractmethod property

controllers: dict[str, Controller]

One or more controllers for the robot joints / wheels / etc.

kinematics abstractmethod property

kinematics: MlSpacesKinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace abstractmethod property

namespace: str

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics abstractmethod property

parallel_kinematics: ParallelKinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view abstractmethod property

robot_view: RobotView

robot view for interfacing with joints / links / actuators

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/abstract.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: BaseRobotConfig,
    spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    """
    Add a robot to a scene, taking care of any robot-specific considerations.

    Args:
        robot_config: The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)
        spec: The scene to insert the robot into
        prefix: The prefix to use for the robot, i.e. the namespace
        pos: The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.
        quat: The quaternion to use for the robot, in [w, x, y, z] format.
        randomize_textures: Whether to randomize the textures of the robot, if applicable. Not supported by all robots.
        strip_meshes: If True, remove all mesh geometries before insertion (for kinematics-only models).
    """
    pos = pos + [0.0] if len(pos) == 2 else pos
    robot_spec = cls._load_robot_spec(robot_config, strip_meshes=strip_meshes)
    robot_root_name = cls.robot_model_root_name()
    attach_frame = spec.worldbody.add_frame(pos=pos, quat=quat)

    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: BaseRobotConfig):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    # TODO: control overrides will break if non-robot actuators are present
    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

    force_limit = robot_config.force_limit
    if force_limit is not None:
        log.debug(f"Applying force limits to robot {cls.robot_model_root_name()}")
        assert len(force_limit) <= len(actuators), (
            "number of force limits cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(force_limit)]):
            actuator.forcelimited = 1
            actuator.forcerange[:] = [-force_limit[i], force_limit[i]]

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/abstract.py

def compute_control(self) -> None:
    """
    Compute and set the control inputs to the robot actuators based on the
    current state and the targets set by the user.
    Must be called after update_control().
    """
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

create_robot_sensors

create_robot_sensors() -> list[Sensor]

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/abstract.py

def create_robot_sensors(self) -> list[Sensor]:
    """Get robot-specific sensors that should be registerd with a Task's sensor suite."""
    return []

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/abstract.py

def get_arm_move_group_ids(self) -> list[str]:
    """Get the move group IDs that should have TCP-bounded noise applied.

    Override in subclass to specify which move groups are arms.
    Default returns empty list (no noise applied).
    """
    return []

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset abstractmethod

reset() -> None

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/abstract.py

@abc.abstractmethod
def reset(self) -> None:
    """Reset the robot to its initial state or a provided state."""
    raise NotImplementedError

robot_model_root_name abstractmethod staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/abstract.py

@staticmethod
@abstractmethod
def robot_model_root_name() -> str:
    """The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion."""
    raise NotImplementedError

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, ndarray]) -> None

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/abstract.py

def update_control(self, action_command_dict: dict[str, np.ndarray]) -> None:
    """
    Update the control targets to the robot based on the provided action commands.
    Does not set the control inputs to the robot actuators. See compute_control().

    Args:
        action_command_dict: Dictionary containing action commands for the robot
                             based on the move groups ids to be used.
    """
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

bimanual_yam

Classes:

Name	Description
BimanualYamRobot

BimanualYamRobot

BimanualYamRobot(mj_data: MjData, config: BaseRobotConfig)

Bases: Robot

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	Bimanual YAM has two independent arms.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The root body name in the bimanual_yam.xml.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers	dict[str, Controller]	One or more controllers for the robot joints / wheels / etc.
kinematics		kinematic solver for the robot
mj_data
mj_model
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		parallel kinematic solver for the robot
robot_config
robot_view		robot view for interfacing with joints / links / actuators

Source code in molmo_spaces/robots/bimanual_yam.py

def __init__(self, mj_data: mj.MjData, config: BaseRobotConfig) -> None:
    super().__init__(mj_data, config)

    assert config.robot_view_factory, (
        "Something went wrong, 'robot_view_factory' shouldn't be None"
    )
    self._robot_view = config.robot_view_factory(mj_data, config.robot_namespace)
    self._kinematics = MlSpacesKinematics(config)

    # Use DummyParallelKinematics for batch IK (wraps the MlSpacesKinematics)
    # Default to left arm for parallel kinematics
    self._parallel_kinematics = DummyParallelKinematics(config, self._kinematics)

    arm_command_mode = config.command_mode.get("arm", "joint_position")
    if arm_command_mode == "joint_rel_position":
        arm_controller_cls = JointRelPosController
    else:
        arm_controller_cls = JointPosController

    gripper_command_mode = config.command_mode.get("gripper", "joint_position")
    if gripper_command_mode == "joint_rel_position":
        gripper_controller_cls = JointRelPosController
    else:
        gripper_controller_cls = JointPosController

    self._controllers: dict[str, Controller] = {
        "left_arm": arm_controller_cls(self._robot_view.get_move_group("left_arm")),
        "right_arm": arm_controller_cls(self._robot_view.get_move_group("right_arm")),
        "left_gripper": gripper_controller_cls(self._robot_view.get_move_group("left_gripper")),
        "right_gripper": gripper_controller_cls(
            self._robot_view.get_move_group("right_gripper")
        ),
    }

controllers property

controllers: dict[str, Controller]

One or more controllers for the robot joints / wheels / etc.

kinematics property

kinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

robot view for interfacing with joints / links / actuators

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BimanualYamRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/bimanual_yam.py

@classmethod
def add_robot_to_scene(  # pyright: ignore[reportIncompatibleMethodOverride] # ty: ignore[invalid-method-override]
    cls,
    robot_config: BimanualYamRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    add_base = robot_config.base_size is not None
    pos = pos + [0.0] if len(pos) == 2 else pos

    # Create a mocap body to control the robot base pose
    robot_body = spec.worldbody.add_body(name=f"{prefix}base", pos=pos, quat=quat, mocap=True)

    if add_base:
        assert robot_config.base_size, (
            "If using 'base' must provide 'base_size' in configuration"
        )
        base_height = robot_config.base_size[2]

        # Create a base material (plain dark wood color)
        material_name = f"{prefix}robot_base_material"
        spec.add_material(name=material_name, rgba=[0.3, 0.2, 0.1, 1.0])

        # Add base geometry (platform)
        robot_body.add_geom(
            type=mj.mjtGeom.mjGEOM_BOX,
            size=[x / 2 for x in robot_config.base_size],
            pos=[0, 0, base_height / 2],
            material=material_name,
            group=0,  # Visual group
        )
        attach_frame = robot_body.add_frame(pos=[0, 0, base_height])
    else:
        attach_frame = robot_body.add_frame()

    robot_spec = cls._load_robot_spec(robot_config, strip_meshes=strip_meshes)
    robot_root_name = cls.robot_model_root_name()
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: BaseRobotConfig):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    # TODO: control overrides will break if non-robot actuators are present
    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

    force_limit = robot_config.force_limit
    if force_limit is not None:
        log.debug(f"Applying force limits to robot {cls.robot_model_root_name()}")
        assert len(force_limit) <= len(actuators), (
            "number of force limits cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(force_limit)]):
            actuator.forcelimited = 1
            actuator.forcerange[:] = [-force_limit[i], force_limit[i]]

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/abstract.py

def compute_control(self) -> None:
    """
    Compute and set the control inputs to the robot actuators based on the
    current state and the targets set by the user.
    Must be called after update_control().
    """
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

create_robot_sensors

create_robot_sensors()

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/bimanual_yam.py

def create_robot_sensors(self):
    return super().create_robot_sensors() + [
        TCPPoseSensor(uuid="tcp_pose_left", gripper_mg_id="left_gripper"),
        TCPPoseSensor(uuid="tcp_pose_right", gripper_mg_id="right_gripper"),
    ]

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Bimanual YAM has two independent arms.

Source code in molmo_spaces/robots/bimanual_yam.py

def get_arm_move_group_ids(self) -> list[str]:
    """Bimanual YAM has two independent arms."""
    return ["left_arm", "right_arm"]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset() -> None

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/bimanual_yam.py

def reset(self) -> None:
    for mg_id, default_pos in self.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = np.array(default_pos)

robot_model_root_name staticmethod

robot_model_root_name() -> str

The root body name in the bimanual_yam.xml.

Source code in molmo_spaces/robots/bimanual_yam.py

@staticmethod
def robot_model_root_name() -> str:
    """The root body name in the bimanual_yam.xml."""
    return "bimanual_base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, ndarray]) -> None

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/abstract.py

def update_control(self, action_command_dict: dict[str, np.ndarray]) -> None:
    """
    Update the control targets to the robot based on the provided action commands.
    Does not set the control inputs to the robot actuators. See compute_control().

    Args:
        action_command_dict: Dictionary containing action commands for the robot
                             based on the move groups ids to be used.
    """
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

find_gripper_finger_range

Script to find the inter finger distance range for all gripper move groups in a robot. This is useful for adding new robots to MolmoSpaces.

Functions:

Name	Description
get_args
main

get_args

get_args()

Source code in molmo_spaces/robots/find_gripper_finger_range.py

def get_args():
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument(
        "robot_config_module", type=str, help="Python module which contains the robot config class"
    )
    parser.add_argument("robot_config_class", type=str, help="Class name of the robot config")
    parser.add_argument(
        "--settle-time",
        type=float,
        default=2.0,
        help="Amount of simulation time to let the gripper go to the target (default: 2.0s)",
    )
    return parser.parse_args()

main

main()

Source code in molmo_spaces/robots/find_gripper_finger_range.py

def main():
    args = get_args()
    robot_config_module = importlib.import_module(args.robot_config_module)
    robot_config_class = getattr(robot_config_module, args.robot_config_class)

    robot_config: BaseRobotConfig = robot_config_class()
    assert robot_config.robot_cls is not None, "Must have provided a valid robot-class"
    assert robot_config.robot_view_factory is not None, (
        "Must have provided a valid robot-view-factory"
    )

    spec = MjSpec()
    robot_config.robot_cls.add_robot_to_scene(
        robot_config,
        spec,
        prefix=robot_config.robot_namespace,
        pos=[0.0, 0.0, 0.0],
        quat=[1.0, 0.0, 0.0, 0.0],
    )

    model: MjModel = spec.compile()
    data = MjData(model)

    robot_view = robot_config.robot_view_factory(data, robot_config.robot_namespace)

    mujoco.mj_forward(model, data)
    gripper_names = robot_view.get_gripper_movegroup_ids()
    gripper_groups = [robot_view.get_gripper(gripper_name) for gripper_name in gripper_names]

    for gripper_group in gripper_groups:
        gripper_group.set_gripper_ctrl_open(True)

    mujoco.mj_step(model, data, round(1.0 / model.opt.timestep))

    dists_hi = [gripper.inter_finger_dist for gripper in gripper_groups]

    for gripper in gripper_groups:
        gripper.set_gripper_ctrl_open(False)

    mujoco.mj_step(model, data, round(1.0 / model.opt.timestep))

    dists_lo = [gripper.inter_finger_dist for gripper in gripper_groups]

    print("Gripper finger ranges:")
    for gripper_name, dist_lo, dist_hi in zip(gripper_names, dists_lo, dists_hi):
        print(f"\tGripper '{gripper_name}': {dist_lo:.3f} - {dist_hi:.3f}")

floating_robotiq

Classes:

Name	Description
FloatingRobotiqRobot

FloatingRobotiqRobot

FloatingRobotiqRobot(mj_data: MjData, config: BaseRobotConfig)

Bases: FloatingRUMRobot

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers		One or more controllers for the robot joints / wheels / etc.
kinematics		kinematic solver for the robot
mj_data
mj_model
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		parallel kinematic solver for the robot
robot_config
robot_view		robot view for interfacing with joints / links / actuators

Source code in molmo_spaces/robots/floating_rum.py

def __init__(self, mj_data: mj.MjData, config: BaseRobotConfig):
    super().__init__(mj_data, config)

    assert config.robot_view_factory, (
        "Something went wrong, 'robot_view_factory' shouldn't be None"
    )
    self._robot_view = config.robot_view_factory(mj_data, config.robot_namespace)
    self._kinematics = FloatingRUMKinematics(config)
    self._parallel_kinematics = DummyParallelKinematics(config, self._kinematics)
    self._last_cmd_action: dict[str, np.ndarray] | None = None

controllers property

controllers

One or more controllers for the robot joints / wheels / etc.

kinematics property

kinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

robot view for interfacing with joints / links / actuators

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/floating_robotiq.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: BaseRobotConfig,
    spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    pos = pos + [0.0] if len(pos) == 2 else pos
    # call grandparent class method (Robot) directly, skipping FloatingRUMRobot
    Robot.add_robot_to_scene.__func__(
        cls,
        robot_config=robot_config,
        spec=spec,
        prefix=prefix,
        pos=pos,
        quat=quat,
        strip_meshes=strip_meshes,
    )

    # add target pose body and weld to base
    target_body_name = f"{prefix}target_ee_pose"
    spec.worldbody.add_body(name=target_body_name, pos=pos, quat=quat, mocap=True)
    eq = spec.add_equality()
    eq.name1 = target_body_name
    eq.name2 = f"{prefix}{cls.robot_model_root_name()}"
    # Stronger weld constraint: lower solref[0] = stiffer (0.001 is much stiffer than 0.02)
    # This makes the mocap body follow the target pose more precisely, allowing better lifting

    # best for lift to pick objects strongly
    eq.solref = np.array([0.001, 1])
    eq.solimp = np.array([0.99, 0.99, 0.001, 1, 2])

    # best for articulation where it's compliant``
    # eq.solref = np.array([0.01, 1])
    # eq.solimp = np.array([0.8, 0.8, 0.05, 1, 2])

    # in between?
    # eq.solref = np.array([0.0025, 1])
    # eq.solimp = np.array([0.99, 0.85, 0.01, 1, 2])

    eq.objtype = mjtObj.mjOBJ_BODY
    eq.type = mjtEq.mjEQ_WELD

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: BaseRobotConfig):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    # TODO: control overrides will break if non-robot actuators are present
    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

    force_limit = robot_config.force_limit
    if force_limit is not None:
        log.debug(f"Applying force limits to robot {cls.robot_model_root_name()}")
        assert len(force_limit) <= len(actuators), (
            "number of force limits cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(force_limit)]):
            actuator.forcelimited = 1
            actuator.forcerange[:] = [-force_limit[i], force_limit[i]]

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/floating_rum.py

def compute_control(self) -> None:
    assert self._last_cmd_action is not None
    for mg_id, ctrl in self._last_cmd_action.items():
        # Some policies (e.g. AStarPlannerPolicy) include non-move-group control
        # flags like "done" in the action dict; only "done": True is meant to be
        # consumed (by BaseMujocoTask._apply_action, which pops it beforehand).
        if mg_id not in self._robot_view.move_group_ids():
            continue
        if ctrl is not None:
            move_group = self._robot_view.get_move_group(mg_id)
            if mg_id == "base" and len(ctrl) == 3:
                # Nav policies (e.g. AStarPlannerPolicy) command the mobile base as
                # a planar [x, y, theta] waypoint. Translate that into this floating
                # base's [x, y, z, qw, qx, qy, qz] ctrl, keeping the current height
                # and a level (roll=pitch=0) orientation.
                current_z = move_group.ctrl[2]
                quat = R.from_euler("z", ctrl[2]).as_quat(scalar_first=True)
                ctrl = np.array([ctrl[0], ctrl[1], current_z, *quat])
            move_group.ctrl = ctrl

create_robot_sensors

create_robot_sensors()

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/floating_rum.py

def create_robot_sensors(self):
    return super().create_robot_sensors() + [
        TCPPoseSensor(uuid="tcp_pose"),
    ]

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/abstract.py

def get_arm_move_group_ids(self) -> list[str]:
    """Get the move group IDs that should have TCP-bounded noise applied.

    Override in subclass to specify which move groups are arms.
    Default returns empty list (no noise applied).
    """
    return []

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset()

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/floating_rum.py

def reset(self):
    self._last_cmd_action = None
    for mg_id, default_pos in self.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = np.array(default_pos)

robot_model_root_name staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/floating_rum.py

@staticmethod
def robot_model_root_name() -> str:
    return "base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, Any])

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/floating_rum.py

def update_control(self, action_command_dict: dict[str, Any]):
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)
    self._last_cmd_action = action_command_dict

floating_rum

Classes:

Name	Description
FloatingRUMRobot

FloatingRUMRobot

FloatingRUMRobot(mj_data: MjData, config: BaseRobotConfig)

Bases: Robot

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers		One or more controllers for the robot joints / wheels / etc.
kinematics		kinematic solver for the robot
mj_data
mj_model
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		parallel kinematic solver for the robot
robot_config
robot_view		robot view for interfacing with joints / links / actuators

Source code in molmo_spaces/robots/floating_rum.py

def __init__(self, mj_data: mj.MjData, config: BaseRobotConfig):
    super().__init__(mj_data, config)

    assert config.robot_view_factory, (
        "Something went wrong, 'robot_view_factory' shouldn't be None"
    )
    self._robot_view = config.robot_view_factory(mj_data, config.robot_namespace)
    self._kinematics = FloatingRUMKinematics(config)
    self._parallel_kinematics = DummyParallelKinematics(config, self._kinematics)
    self._last_cmd_action: dict[str, np.ndarray] | None = None

controllers property

controllers

One or more controllers for the robot joints / wheels / etc.

kinematics property

kinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

robot view for interfacing with joints / links / actuators

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/floating_rum.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: BaseRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    pos = pos + [0.0] if len(pos) == 2 else pos
    super().add_robot_to_scene(
        robot_config=robot_config,
        spec=spec,
        prefix=prefix,
        pos=pos,
        quat=quat,
        randomize_textures=randomize_textures,
        strip_meshes=strip_meshes,
    )

    target_body_name = f"{prefix}target_ee_pose"
    spec.worldbody.add_body(name=target_body_name, pos=pos, quat=quat, mocap=True)
    spec.add_equality(
        type=mj.mjtEq.mjEQ_WELD,
        name1=target_body_name,
        name2=f"{prefix}{cls.robot_model_root_name()}",
        solref=[0.02, 1],
        solimp=[0.9, 0.95, 0.0, 1, 2],
        objtype=mj.mjtObj.mjOBJ_BODY,
    )

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: BaseRobotConfig):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    # TODO: control overrides will break if non-robot actuators are present
    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

    force_limit = robot_config.force_limit
    if force_limit is not None:
        log.debug(f"Applying force limits to robot {cls.robot_model_root_name()}")
        assert len(force_limit) <= len(actuators), (
            "number of force limits cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(force_limit)]):
            actuator.forcelimited = 1
            actuator.forcerange[:] = [-force_limit[i], force_limit[i]]

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/floating_rum.py

def compute_control(self) -> None:
    assert self._last_cmd_action is not None
    for mg_id, ctrl in self._last_cmd_action.items():
        # Some policies (e.g. AStarPlannerPolicy) include non-move-group control
        # flags like "done" in the action dict; only "done": True is meant to be
        # consumed (by BaseMujocoTask._apply_action, which pops it beforehand).
        if mg_id not in self._robot_view.move_group_ids():
            continue
        if ctrl is not None:
            move_group = self._robot_view.get_move_group(mg_id)
            if mg_id == "base" and len(ctrl) == 3:
                # Nav policies (e.g. AStarPlannerPolicy) command the mobile base as
                # a planar [x, y, theta] waypoint. Translate that into this floating
                # base's [x, y, z, qw, qx, qy, qz] ctrl, keeping the current height
                # and a level (roll=pitch=0) orientation.
                current_z = move_group.ctrl[2]
                quat = R.from_euler("z", ctrl[2]).as_quat(scalar_first=True)
                ctrl = np.array([ctrl[0], ctrl[1], current_z, *quat])
            move_group.ctrl = ctrl

create_robot_sensors

create_robot_sensors()

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/floating_rum.py

def create_robot_sensors(self):
    return super().create_robot_sensors() + [
        TCPPoseSensor(uuid="tcp_pose"),
    ]

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/abstract.py

def get_arm_move_group_ids(self) -> list[str]:
    """Get the move group IDs that should have TCP-bounded noise applied.

    Override in subclass to specify which move groups are arms.
    Default returns empty list (no noise applied).
    """
    return []

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset()

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/floating_rum.py

def reset(self):
    self._last_cmd_action = None
    for mg_id, default_pos in self.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = np.array(default_pos)

robot_model_root_name staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/floating_rum.py

@staticmethod
def robot_model_root_name() -> str:
    return "base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, Any])

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/floating_rum.py

def update_control(self, action_command_dict: dict[str, Any]):
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)
    self._last_cmd_action = action_command_dict

franka

Classes:

Name	Description
FrankaRobot

Attributes:

Name	Type	Description
log

log module-attribute

log = logging.getLogger(__name__)

FrankaRobot

FrankaRobot(mj_data: MjData, config: BaseRobotConfig)

Bases: Robot

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_base_material
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	Franka has a single arm move group.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
merge_gripper_pads	Fuse each finger's two pad boxes into one, so they share no seam.
randomize_robot_textures
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers	dict[str, Controller]	One or more controllers for the robot joints / wheels / etc.
kinematics		kinematic solver for the robot
mj_data
mj_model
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		parallel kinematic solver for the robot
robot_config
robot_view		robot view for interfacing with joints / links / actuators

Source code in molmo_spaces/robots/franka.py

def __init__(self, mj_data: mj.MjData, config: BaseRobotConfig) -> None:
    super().__init__(mj_data, config)

    assert config.robot_view_factory, (
        "Something went wrong, 'robot_view_factory' shouldn't be None"
    )
    self._robot_view = config.robot_view_factory(mj_data, config.robot_namespace)
    self._kinematics = MlSpacesKinematics(config)

    # TODO(wilbert): use this one only if not in Darwin|MacOS, as warp works poorly when doing
    # emulation on non-nvidia hardware
    self._parallel_kinematics = SimpleWarpKinematics(config)
    arm_controller_cls = (
        JointPosController
        if config.command_mode == {} or config.command_mode["arm"] == "joint_position"
        else JointRelPosController
    )
    self._controllers: dict[str, Controller] = {
        "arm": arm_controller_cls(self._robot_view.get_move_group("arm")),
        "gripper": JointPosController(self._robot_view.get_move_group("gripper")),
    }

controllers property

controllers: dict[str, Controller]

One or more controllers for the robot joints / wheels / etc.

kinematics property

kinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

robot view for interfacing with joints / links / actuators

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: FrankaRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/franka.py

@classmethod
def add_robot_to_scene(  # pyright: ignore[reportIncompatibleMethodOverride] # ty: ignore[invalid-method-override]
    cls,
    robot_config: FrankaRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    # TODO(wilbert): this part is iffy, we shouldn't expect to receive either both vec3 and vec2
    # here. The caller should just send a vec3 and if he just wants xy then the last element
    # should be just 0. We could maybe change list[float] to tuple[float,float,float] and so on
    pos_xyz = pos + [0.0] if len(pos) == 2 else pos.copy()

    material_name = cls.create_robot_base_material(
        robot_config, spec, prefix, randomize_textures
    )

    robot_body = spec.worldbody.add_body(
        name=f"{prefix}base",
        pos=pos_xyz,
        quat=quat,
        mocap=True,
    )
    if robot_config.base_size is not None:
        assert robot_config.base_size, (
            "If using 'base' must provide 'base_size' in configuration"
        )
        base_height = robot_config.base_size[2]

        # Add base geometry (wooden platform)
        robot_body.add_geom(
            type=mj.mjtGeom.mjGEOM_BOX,
            size=[x / 2 for x in robot_config.base_size],
            pos=[0, 0, base_height / 2],
            material=material_name,
            group=0,  # Visual group
        )
        attach_frame = robot_body.add_frame(pos=[0, 0, base_height])
    else:
        attach_frame = robot_body.add_frame()

    robot_spec = cls._load_robot_spec(robot_config, strip_meshes=strip_meshes)
    cls.merge_gripper_pads(robot_spec)

    if randomize_textures:
        cls.randomize_robot_textures(robot_config, spec, prefix, robot_spec)

    robot_root_name = cls.robot_model_root_name()
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: BaseRobotConfig):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    # TODO: control overrides will break if non-robot actuators are present
    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

    force_limit = robot_config.force_limit
    if force_limit is not None:
        log.debug(f"Applying force limits to robot {cls.robot_model_root_name()}")
        assert len(force_limit) <= len(actuators), (
            "number of force limits cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(force_limit)]):
            actuator.forcelimited = 1
            actuator.forcerange[:] = [-force_limit[i], force_limit[i]]

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/abstract.py

def compute_control(self) -> None:
    """
    Compute and set the control inputs to the robot actuators based on the
    current state and the targets set by the user.
    Must be called after update_control().
    """
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

create_robot_base_material classmethod

create_robot_base_material(robot_config: FrankaRobotConfig, spec: MjSpec, prefix: str, randomize_base_texture: bool) -> str

Source code in molmo_spaces/robots/franka.py

@classmethod
def create_robot_base_material(
    cls,
    robot_config: FrankaRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    randomize_base_texture: bool,
) -> str:
    texture_dir = robot_config.get_robot_dir() / "assets" / "base_textures"
    assert texture_dir.is_dir(), f"Texture directory {texture_dir} does not exist"
    texture_path: Path | None = None
    if randomize_base_texture:
        texture_paths = list(texture_dir.glob("*.png"))
        texture_paths.sort(key=lambda x: x.name)
        assert len(texture_paths) > 0, f"No robot base texture paths found in {texture_dir}"
        log.debug(f"Found {len(texture_paths)} robot base texture paths")
        texture_path = random.choice(texture_paths)
    else:
        texture_path = texture_dir / "DarkWood2.png"
        assert texture_path.is_file(), f"Default texture {texture_path} does not exist"

    texture_name = f"{prefix}robot_base_texture"
    spec.add_texture(
        name=texture_name,
        type=mj.mjtTexture.mjTEXTURE_CUBE,
        file=str(texture_path),
    )
    log.debug(f"Successfully created texture from {texture_path}")

    material_name = f"{prefix}robot_base_material"
    robot_base_mat = spec.add_material(name=material_name)
    robot_base_mat.textures[mj.mjtTextureRole.mjTEXROLE_RGB] = texture_name
    log.debug(f"Successfully created material {material_name}")
    return material_name

create_robot_sensors

create_robot_sensors()

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/franka.py

def create_robot_sensors(self):
    return super().create_robot_sensors() + [
        TCPPoseSensor(uuid="tcp_pose"),
    ]

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Franka has a single arm move group.

Source code in molmo_spaces/robots/franka.py

def get_arm_move_group_ids(self) -> list[str]:
    """Franka has a single arm move group."""
    return ["arm"]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

merge_gripper_pads classmethod

merge_gripper_pads(robot_spec: MjSpec) -> int

Fuse each finger's two pad boxes into one, so they share no seam.

Each Robotiq finger carries its pad as two stacked boxes on one body, pad1 and pad2. An object thin enough to penetrate that boundary ends up inside both boxes at once, and each pushes it out of itself and therefore into the other. The surviving collider is pad2, which is the geom the robot views identify each finger by. Returns how many fingers were merged.

Source code in molmo_spaces/robots/franka.py

@classmethod
def merge_gripper_pads(cls, robot_spec: mj.MjSpec) -> int:
    """Fuse each finger's two pad boxes into one, so they share no seam.

    Each Robotiq finger carries its pad as two stacked boxes on one body,
    pad1 and pad2. An object thin enough to penetrate that boundary ends up
    inside both boxes at once, and each pushes it out of itself and therefore
    into the other. The surviving collider is ``pad2``, which is the geom the
    robot views identify each finger by. Returns how many fingers were merged.
    """
    pads: dict[str, dict[str, mj.MjsGeom]] = {}
    for geom in robot_spec.geoms:
        if geom.name.endswith("_pad1") or geom.name.endswith("_pad2"):
            pads.setdefault(geom.name[:-1], {})[geom.name[-1]] = geom
    merged = 0
    for finger, pair in sorted(pads.items()):
        if set(pair) != {"1", "2"}:
            continue
        first, second = pair["1"], pair["2"]
        low = min(first.pos[2] - first.size[2], second.pos[2] - second.size[2])
        high = max(first.pos[2] + first.size[2], second.pos[2] + second.size[2])
        second.pos = np.array([second.pos[0], second.pos[1], (low + high) / 2.0])
        second.size = np.array([second.size[0], second.size[1], (high - low) / 2.0])
        # Kept as geometry but no longer colliding: the merged box already
        # covers its span, and two colliders over one span is the whole bug.
        first.contype = 0
        first.conaffinity = 0
        merged += 1
        log.debug("merged gripper pads for %s into %s2", finger, finger)
    return merged

randomize_robot_textures classmethod

randomize_robot_textures(robot_config: FrankaRobotConfig, spec: MjSpec, prefix: str, robot_spec: MjSpec)

Source code in molmo_spaces/robots/franka.py

@classmethod
def randomize_robot_textures(
    cls,
    robot_config: FrankaRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    robot_spec: mj.MjSpec,
):
    if random.random() > robot_config.perturb_texture_probability:
        log.info(f"Skipping texture randomization for robot '{robot_config.name}'")
        return

    perturbed_materials: dict[str, str] = {}
    for material in robot_spec.materials:
        material: mj.MjsMaterial
        is_rgb_mat = all(
            material.textures[i] == "" for i in range(mj.mjtTextureRole.mjNTEXROLE)
        )
        if not is_rgb_mat:
            continue

        speckle_img = speckle_texture(material.rgba[:3])
        buffer = BytesIO()
        speckle_img.save(buffer, format="PNG")
        buffer.seek(0)

        tex_name = f"{material.name}_perturbed_tex"
        mat_name = f"{material.name}_perturbed"
        fn = f"{prefix}{tex_name}.png".replace("/", "__")
        spec.assets[fn] = buffer.getvalue()
        robot_spec.add_texture(name=tex_name, type=mj.mjtTexture.mjTEXTURE_2D, file=fn)
        perturbed_mat = robot_spec.add_material(name=mat_name)
        perturbed_mat.textures[mj.mjtTextureRole.mjTEXROLE_RGB] = tex_name
        perturbed_materials[material.name] = mat_name

    def set_material(body: mj.MjsBody):
        for geom in body.geoms:
            geom: mj.MjsGeom
            if geom.material in perturbed_materials:
                log.debug(
                    f"Setting material {geom.material} to {perturbed_materials[geom.material]} "
                    f"for geom '{geom.name}' in body '{body.name}'"
                )
                geom.material = perturbed_materials[geom.material]
        for child in body.bodies:
            set_material(child)

    robot_body = robot_spec.body(cls.robot_model_root_name())
    set_material(robot_body)
    log.info(f"Successfully randomized robot textures for robot '{robot_config.name}'")

reset

reset() -> None

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/franka.py

def reset(self) -> None:
    for mg_id, default_pos in self.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = np.array(default_pos)

robot_model_root_name staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/franka.py

@staticmethod
def robot_model_root_name() -> str:
    return "fr3_link0"

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, ndarray]) -> None

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/abstract.py

def update_control(self, action_command_dict: dict[str, np.ndarray]) -> None:
    """
    Update the control targets to the robot based on the provided action commands.
    Does not set the control inputs to the robot actuators. See compute_control().

    Args:
        action_command_dict: Dictionary containing action commands for the robot
                             based on the move groups ids to be used.
    """
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

g1

The G1 humanoid robot: g1_molmo's components/robot.py on molmo_spaces' Robot interface, verified byte-identical against the gold pick rollout (projects/fetchman/scripts/). G1Config constructs it through from_mj_data; its whole-body controller is controllers/g1_wbc.py.

Classes:

Name	Description
G1Robot
G1RobotView	Pose/contact/visibility helpers for the G1 (the reference stack's

Attributes:

Name	Type	Description
ACTUATOR_NAME_MAP
DEFAULT_QPOS
GRIPPER_CLOSED
GRIPPER_OPEN
HAND_ARM_JOINTS
HAND_SITE
IK_DT
IK_HEIGHT_DAMPING
JOINT_NAMES
MAX_LINEAR_VEL
MAX_YAW_RATE
MIN_LINEAR_VEL
MIN_YAW_RATE
NUM_LEGS_WAIST_TARGET_DIMS
PELVIS_FORWARD_OFFSET
PREFIX
ROOT_BODY
STANDING_HEIGHT
VELOCITY_DEADBAND
WAIST_JOINTS
XML_PATH
YAW_GATE_THRESHOLD

ACTUATOR_NAME_MAP module-attribute

ACTUATOR_NAME_MAP = {'right_Joint1_1': 'right_grip'}

DEFAULT_QPOS module-attribute

DEFAULT_QPOS = _resolve_defaults()

GRIPPER_CLOSED module-attribute

GRIPPER_CLOSED = 0.0245

GRIPPER_OPEN module-attribute

GRIPPER_OPEN = -0.0222

HAND_ARM_JOINTS module-attribute

HAND_ARM_JOINTS = {side: _hand_joints(side) for side in ('left', 'right')}

HAND_SITE module-attribute

HAND_SITE = {side: f'{side}_grasp' for side in ('left', 'right')}

IK_DT module-attribute

IK_DT = 0.01

IK_HEIGHT_DAMPING module-attribute

IK_HEIGHT_DAMPING = 500000.0

JOINT_NAMES module-attribute

JOINT_NAMES = ['left_hip_pitch_joint', 'left_hip_roll_joint', 'left_hip_yaw_joint', 'left_knee_joint', 'left_ankle_pitch_joint', 'left_ankle_roll_joint', 'right_hip_pitch_joint', 'right_hip_roll_joint', 'right_hip_yaw_joint', 'right_knee_joint', 'right_ankle_pitch_joint', 'right_ankle_roll_joint', 'waist_yaw_joint', 'waist_roll_joint', 'waist_pitch_joint', 'left_shoulder_pitch_joint', 'left_shoulder_roll_joint', 'left_shoulder_yaw_joint', 'left_elbow_joint', 'left_wrist_roll_joint', 'left_wrist_pitch_joint', 'left_wrist_yaw_joint', 'right_shoulder_pitch_joint', 'right_shoulder_roll_joint', 'right_shoulder_yaw_joint', 'right_elbow_joint', 'right_wrist_roll_joint', 'right_wrist_pitch_joint', 'right_wrist_yaw_joint', 'right_Joint1_1']

MAX_LINEAR_VEL module-attribute

MAX_LINEAR_VEL = 0.5

MAX_YAW_RATE module-attribute

MAX_YAW_RATE = 0.5

MIN_LINEAR_VEL module-attribute

MIN_LINEAR_VEL = 0.15

MIN_YAW_RATE module-attribute

MIN_YAW_RATE = 0.15

NUM_LEGS_WAIST_TARGET_DIMS module-attribute

NUM_LEGS_WAIST_TARGET_DIMS = 7

PELVIS_FORWARD_OFFSET module-attribute

PELVIS_FORWARD_OFFSET = 0.05

PREFIX module-attribute

PREFIX = 'robot_0/'

ROOT_BODY module-attribute

ROOT_BODY = f'{PREFIX}pelvis'

STANDING_HEIGHT module-attribute

STANDING_HEIGHT = 0.75

VELOCITY_DEADBAND module-attribute

VELOCITY_DEADBAND = 0.08

WAIST_JOINTS module-attribute

WAIST_JOINTS = ('waist_yaw_joint', 'waist_roll_joint', 'waist_pitch_joint')

XML_PATH module-attribute

XML_PATH = str(ASSETS_DIR / 'robots/g1/g1_dex.xml')

YAW_GATE_THRESHOLD module-attribute

YAW_GATE_THRESHOLD = np.radians(30)

G1Robot

G1Robot(data: MjData, robot_config: G1Config, env=None, low_level=None, namespace: str = PREFIX, xml_path: str = XML_PATH, gripper_friction: tuple[float, float, float] | None = None)

Bases: Robot

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
advance_control_clock	Advance the WBC's decimation clock by one tick. The reference policy
apply_action_noise	Apply action noise to the commanded action.
apply_arm_pose	Apply a {joint_name: qpos_value} dict produced by IK to the scene qpos.
apply_control_overrides	Actuator/gain configuration is done at runtime by
apply_upper_pose
check_object_visibility
close	Free the lazily-created visibility renderer (and its GL/EGL context).
compute_control	Robot.compute_control, dispatching in G1WholeBodyController.execute_action's
create_robot_sensors	Robot.create_robot_sensors: this robot's own state sensors, which
execute_action
from_mj_data	molmo_spaces' (mj_data, robot_config) robot-factory constructor;
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
get_xy
get_yaw
has_bad_contacts
kinematics_wbc	gold's G1Controller._solve_ik_wbc: mink IK on a standalone robot-only
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
nav_action	A full flat-15 action that walks toward waypoint while holding the
pelvis_height
place
reset	Robot ABC reset: DEFAULT_QPOS (the reference stack's settled standing
robot_model_root_name	Robot ABC hook used by the scene builder to find the robot's root
sample_upper_pose
set_defaults
set_env	Wires the low-level controller to env (mj model opt overrides,
set_joint_pos	Set all the robot's joint positions to the specified values.
set_pose
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
solve_scene_ik	Full-scene mink IK (gold's GraspPolicy._solve_ik) on the live scene
state_is_finite
update_control	Robot.update_control: a move-group dict, whose "legs_waist" entry is
waypoint_to_velocity_target	Absolute [x, y, yaw] waypoint -> the WBC's [vx, vy, yaw_rate] base
zero_velocities

Attributes:

Name	Type	Description
act_ids
controllers		{move_group_name: Controller}, matching molmo_spaces/robots/
data
kinematics		Robot ABC's kinematics solver. Built lazily: the reference stack
left_gripper_aid
mj_data
mj_model
model
n_substeps
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		Robot ABC's parallel kinematics. DummyParallelKinematics matches
right_gripper_aid
robot_config
robot_view		Robot ABC's RobotView: the MoveGroup view of robot_views/g1_view.py

Source code in molmo_spaces/robots/g1.py

def __init__(
    self,
    data: mj.MjData,
    robot_config: G1Config,
    env=None,
    low_level=None,
    namespace: str = PREFIX,
    xml_path: str = XML_PATH,
    gripper_friction: tuple[float, float, float] | None = None,
):
    # Robot.__init__(mj_data, robot_config) stores its second arg as
    # self.robot_config; this class is constructed with an explicit
    # (model, data) pair by the reference stack, so pass `data` through
    # and let self.robot_config hold the robot config (below reads it as
    # self.robot_config, matching the base class attribute).
    super().__init__(data, robot_config)
    self.model = data.model
    self.data = data
    self._env = env
    # Robot ABC solver properties, built lazily (see `kinematics`).
    self._kinematics = None
    self._parallel_kinematics = None
    # Native MoveGroup-based RobotView, built lazily (see `robot_view`).
    self._native_robot_view = None
    # Set by a policy that advances the WBC gait clock itself (see
    # compute_control).
    self._external_gait_clock = False
    # Namespace/asset path are per-instance rather than the module-level
    # PREFIX/XML_PATH the reference stack hardcoded, matching how every
    # other molmo_spaces Robot takes these from its BaseRobotConfig
    # (robot_namespace / get_robot_xml_path). The module constants remain
    # the defaults, so the reference stack's own call sites are unchanged.
    self._namespace = namespace
    self._xml_path = xml_path
    self._root_body = f"{namespace}pelvis"
    self._gripper_friction = gripper_friction
    # The reference stack sets the physics rate in G1Controller.set_env
    # (m.opt.timestep = 0.005), which molmo_spaces' own env never calls --
    # it constructs robots through the factory and drives them through the
    # task. Apply the configured rate here so a natively-constructed G1
    # gets the 5ms step its WBC was trained at, instead of inheriting the
    # scene default (2ms) and failing BaseMujocoTask's
    # control-dt-divisible-by-sim-dt check.
    if robot_config is not None:
        physics_timestep = getattr(robot_config, "physics_timestep", None)
        if physics_timestep is not None:
            data.model.opt.timestep = physics_timestep

    # The WBC/PD controller is owned by the robot. `low_level` lets a scene
    # reload reuse the instance (its ONNX sessions are expensive); setup()
    # rebinds its index arrays to the new model either way.
    if low_level is None:
        # Deferred import -- controllers/g1_wbc.py imports JOINT_NAMES/
        # DEFAULT_QPOS from this module, so a module-level import here
        # would be circular.
        from molmo_spaces.controllers.g1_wbc import G1WholeBodyController as _LowLevelController

        low_level = _LowLevelController()
    self._low_level = low_level
    self._low_level.setup(self.model, data, prefix=self._namespace)

    self._body_id = mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_BODY, self._root_body)
    if self._body_id < 0:
        raise RuntimeError(f"G1 root body '{self._root_body}' not found")
    self._freejoint_id = mj.mj_name2id(
        self.model, mj.mjtObj.mjOBJ_JOINT, f"{self._namespace}floating_base_joint"
    )

    # kinematics() state -- lazily-built mink.Configuration cache (see
    # kinematics' own docstring) plus the full scene joint-name list
    # _solve_ik reads back solved qpos through.
    self._stj = [
        mj.mj_id2name(self.model, mj.mjtObj.mjOBJ_JOINT, i) for i in range(self.model.njnt)
    ]
    self._fj_scene = self.model.jnt_qposadr[self._freejoint_id].item()
    self._ik_cfg = None
    self._ik_mdl = None
    self._ik_fj_qa = None

    # kinematics_wbc() state -- lazily-built standalone-model mink setup,
    # ported from agents/policy_g1ms.py's G1Controller.setup(). Built on
    # first use rather than here since it's WBC-controller-specific
    # (irrelevant to a G1Robot with no agent attached yet).
    self._wbc_ik_cfg = None
    self._wbc_ik_fj_dof = None
    self._wbc_ik_fj_qa = None
    self._wbc_scene_to_ik_qpos = None
    self._wbc_hand_cfg = None
    self._wbc_waist_qa = None
    self._wbc_posture_task = None
    self._wbc_pelvis_task = None
    self._wbc_limits = None
    self._wbc_self_collision_limit = None

    self._qpos_ids = np.array(
        [
            self.model.jnt_qposadr[
                mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_JOINT, f"{self._namespace}{n}")
            ]
            for n in JOINT_NAMES
        ]
    )
    self._dof_ids = np.array(
        [
            self.model.jnt_dofadr[
                mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_JOINT, f"{self._namespace}{n}")
            ]
            for n in JOINT_NAMES
        ]
    )

    def _find_act(jname):
        act_name = ACTUATOR_NAME_MAP.get(jname, jname)
        aid = mj.mj_name2id(
            self.model, mj.mjtObj.mjOBJ_ACTUATOR, f"{self._namespace}walk_{act_name}"
        )
        if aid < 0:
            aid = mj.mj_name2id(
                self.model, mj.mjtObj.mjOBJ_ACTUATOR, f"{self._namespace}{act_name}"
            )
        return aid

    self.act_ids = np.array([_find_act(n) for n in JOINT_NAMES])

    self.right_gripper_aid = mj.mj_name2id(
        self.model, mj.mjtObj.mjOBJ_ACTUATOR, f"{self._namespace}right_grip"
    )
    self.left_gripper_aid = mj.mj_name2id(
        self.model, mj.mjtObj.mjOBJ_ACTUATOR, f"{self._namespace}left_grip"
    )

    self.n_substeps = max(1, round(0.02 / self.model.opt.timestep))

    # Visibility uses the egocentric head camera so checks match policy POV.
    self._cam_ids = [
        mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_CAMERA, f"{self._namespace}{n}")
        for n in ("head_pov",)
        if mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_CAMERA, f"{self._namespace}{n}") >= 0
    ]
    self._pose_view = G1RobotView(
        self.model,
        data,
        self._body_id,
        self._freejoint_id,
        self._cam_ids,
        self._dof_ids,
        namespace=self._namespace,
    )

    self._apply_solver_overrides()
    self._fix_contacts()

act_ids instance-attribute

act_ids = np.array([_find_act(n) for n in JOINT_NAMES])

controllers property

controllers

{move_group_name: Controller}, matching molmo_spaces/robots/ abstract.py's Robot.controllers.

data instance-attribute

data = data

kinematics property

kinematics

Robot ABC's kinematics solver. Built lazily: the reference stack constructs this robot with no robot_config at all (it drives the arm through solve_scene_ik/kinematics_wbc, never through MlSpacesKinematics), so requiring one up front would break that path. Native callers that do pass a robot_config get the same solver every other molmo_spaces Robot exposes.

left_gripper_aid instance-attribute

left_gripper_aid = mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_ACTUATOR, f'{self._namespace}left_grip')

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

model instance-attribute

model = data.model

n_substeps instance-attribute

n_substeps = max(1, round(0.02 / self.model.opt.timestep))

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

Robot ABC's parallel kinematics. DummyParallelKinematics matches what native G1Robot uses -- G1 has no batched-IK backend.

right_gripper_aid instance-attribute

right_gripper_aid = mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_ACTUATOR, f'{self._namespace}right_grip')

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

Robot ABC's RobotView: the MoveGroup view of robot_views/g1_view.py when constructed with a robot_config (molmo_spaces' factory), else the reference stack's pose helper view. The pass-through methods always use the pose view, whichever is published here.

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/abstract.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: BaseRobotConfig,
    spec: MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    """
    Add a robot to a scene, taking care of any robot-specific considerations.

    Args:
        robot_config: The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)
        spec: The scene to insert the robot into
        prefix: The prefix to use for the robot, i.e. the namespace
        pos: The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.
        quat: The quaternion to use for the robot, in [w, x, y, z] format.
        randomize_textures: Whether to randomize the textures of the robot, if applicable. Not supported by all robots.
        strip_meshes: If True, remove all mesh geometries before insertion (for kinematics-only models).
    """
    pos = pos + [0.0] if len(pos) == 2 else pos
    robot_spec = cls._load_robot_spec(robot_config, strip_meshes=strip_meshes)
    robot_root_name = cls.robot_model_root_name()
    attach_frame = spec.worldbody.add_frame(pos=pos, quat=quat)

    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

advance_control_clock

advance_control_clock() -> None

Advance the WBC's decimation clock by one tick. The reference policy owns the control loop and does this itself; anything else driving the robot has to, or the gait clock never ticks.

Source code in molmo_spaces/robots/g1.py

def advance_control_clock(self) -> None:
    """Advance the WBC's decimation clock by one tick. The reference policy
    owns the control loop and does this itself; anything else driving the
    robot has to, or the gait clock never ticks."""
    self._low_level._step_counter += 1

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_arm_pose

apply_arm_pose(joints_dict)

Apply a {joint_name: qpos_value} dict produced by IK to the scene qpos. Only writes joints in JOINT_NAMES (skips legs/freejoint). Also mirrors right_Joint2_1 to right_Joint1_1.

Source code in molmo_spaces/robots/g1.py

def apply_arm_pose(self, joints_dict):
    """Apply a {joint_name: qpos_value} dict produced by IK to the scene qpos.
    Only writes joints in JOINT_NAMES (skips legs/freejoint). Also mirrors
    right_Joint2_1 to right_Joint1_1."""
    if not joints_dict:
        return
    for i, jname in enumerate(JOINT_NAMES):
        if jname in joints_dict:
            self.data.qpos[self._qpos_ids[i]] = float(joints_dict[jname])
            aid = int(self.act_ids[i])
            if aid >= 0:
                self.data.ctrl[aid] = float(joints_dict[jname])
    j2 = mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_JOINT, f"{self._namespace}right_Joint2_1")
    if j2 >= 0 and "right_Joint1_1" in joints_dict:
        self.data.qpos[self.model.jnt_qposadr[j2]] = float(joints_dict["right_Joint1_1"])

apply_control_overrides classmethod

apply_control_overrides(spec, robot_config)

Actuator/gain configuration is done at runtime by G1WholeBodyController.setup() on the compiled MjModel, so this only carries the spec-time fix that cannot be done later: the MJCF puts the head/mount/ logo visual geoms on group 5, which no renderer in this codebase enables, so the G1 renders headless. Move them to group 2 like the rest of the robot.

Source code in molmo_spaces/robots/g1.py

@classmethod
def apply_control_overrides(cls, spec, robot_config):
    """Actuator/gain configuration is done at runtime by
    G1WholeBodyController.setup() on the compiled MjModel, so this only carries the
    spec-time fix that cannot be done later: the MJCF puts the head/mount/
    logo visual geoms on group 5, which no renderer in this codebase
    enables, so the G1 renders headless. Move them to group 2 like the rest
    of the robot."""
    super().apply_control_overrides(spec, robot_config)
    namespace = robot_config.robot_namespace
    head_mesh_suffixes = ("head_link", "head_mount", "logo_link")
    for body_name in ("torso_link", "head_camera_mount"):
        body = spec.body(f"{namespace}{body_name}")
        if body is None:
            continue
        for geom in body.geoms:
            if geom.group == 5 and geom.meshname.endswith(head_mesh_suffixes):
                geom.group = 2

apply_upper_pose

apply_upper_pose(values)

Source code in molmo_spaces/robots/g1.py

def apply_upper_pose(self, values):
    idx = self._UPPER_RAND_IDX
    self.data.qpos[self._qpos_ids[idx]] = values
    aids = self.act_ids[idx]
    valid = aids >= 0
    if valid.any():
        self.data.ctrl[aids[valid]] = values[valid]
    # Mirror Joint2_1 to Joint1_1 — the equality only enforces during sim, not on init.
    j2 = mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_JOINT, f"{self._namespace}right_Joint2_1")
    if j2 >= 0:
        self.data.qpos[self.model.jnt_qposadr[j2]] = values[self._UPPER_GRIPPER_LOCAL]

check_object_visibility

check_object_visibility(body_id, threshold=2e-05)

Source code in molmo_spaces/robots/g1.py

def check_object_visibility(self, body_id, threshold=0.00002):
    return self._pose_view.check_object_visibility(body_id, threshold=threshold)

close

close()

Free the lazily-created visibility renderer (and its GL/EGL context). Must be called before dropping the robot on scene reload — otherwise the renderer's framebuffer leaks on the render GPU because del-based EGL teardown is unreliable, so VRAM creeps to OOM across reloads.

Source code in molmo_spaces/robots/g1.py

def close(self):
    """Free the lazily-created visibility renderer (and its GL/EGL context).
    Must be called before dropping the robot on scene reload — otherwise the
    renderer's framebuffer leaks on the render GPU because __del__-based EGL
    teardown is unreliable, so VRAM creeps to OOM across reloads."""
    self._pose_view.close()

compute_control

compute_control() -> None

Robot.compute_control, dispatching in G1WholeBodyController.execute_action's controller order (load-bearing: legs_waist torque uses the previous tick's _target_lower before the WBC updates it).

Source code in molmo_spaces/robots/g1.py

def compute_control(self) -> None:
    """Robot.compute_control, dispatching in G1WholeBodyController.execute_action's
    controller order (load-bearing: legs_waist torque uses the previous
    tick's _target_lower before the WBC updates it).
    """
    # Advance the WBC gait clock (ONNX inference every 4th tick) unless the
    # driving policy does it itself (G1PickPlannerPolicy sets
    # _external_gait_clock). Pinned at 0 the network ran every tick and the
    # robot fell over.
    if not self._external_gait_clock:
        self._low_level._step_counter += 1
    data = self.data
    step_counter = self._low_level._step_counter
    for controller in self._low_level._controllers:
        # A controller with no target holds its joints (native policies
        # address only the move groups they care about). legs_waist is
        # exempt: it keeps its command on the owning G1WholeBodyController, so its
        # `target` is always None and skipping it would stop the balancing.
        if controller.target is None and controller.move_group.name != "legs_waist":
            # Hold the current joint positions rather than skipping. These
            # are position actuators (the right arm at kp=2000), so leaving
            # ctrl at whatever was written last drives the arm hard toward a
            # stale target -- enough to throw the robot over during nav.
            # Groups with no joint indices (right_gripper tracks a tendon)
            # have nothing to hold, so leave their ctrl untouched.
            jqpos = controller.move_group.jqpos
            if len(jqpos) == 0:
                continue
            controller.move_group.set_ctrl(data, controller.move_group.joint_pos(data))
            continue
        values = controller.compute_ctrl_inputs(data, step_counter)
        controller.move_group.set_ctrl(data, np.atleast_1d(values))

create_robot_sensors

create_robot_sensors()

Robot.create_robot_sensors: this robot's own state sensors, which BaseMujocoTask (and fetchman's G1Task) append to the task's suite.

Source code in molmo_spaces/robots/g1.py

def create_robot_sensors(self):
    """Robot.create_robot_sensors: this robot's own state sensors, which
    BaseMujocoTask (and fetchman's G1Task) append to the task's suite."""
    from molmo_spaces.env.sensors_g1 import robot_sensors

    return robot_sensors()

execute_action

execute_action(action)

Source code in molmo_spaces/robots/g1.py

def execute_action(self, action):
    return self._low_level.execute_action(action)

from_mj_data classmethod

from_mj_data(mj_data, robot_config) -> G1Robot

molmo_spaces' (mj_data, robot_config) robot-factory constructor; __init__ keeps the reference stack's explicit (model, data) pair.

Source code in molmo_spaces/robots/g1.py

@classmethod
def from_mj_data(cls, mj_data, robot_config) -> G1Robot:
    """molmo_spaces' `(mj_data, robot_config)` robot-factory constructor;
    `__init__` keeps the reference stack's explicit `(model, data)` pair."""
    return cls(
        mj_data,
        namespace=robot_config.robot_namespace,
        xml_path=str(robot_config.get_robot_xml_path()),
        robot_config=robot_config,
    )

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/abstract.py

def get_arm_move_group_ids(self) -> list[str]:
    """Get the move group IDs that should have TCP-bounded noise applied.

    Override in subclass to specify which move groups are arms.
    Default returns empty list (no noise applied).
    """
    return []

get_xy

get_xy()

Source code in molmo_spaces/robots/g1.py

def get_xy(self):
    return self._pose_view.get_xy()

get_yaw

get_yaw()

Source code in molmo_spaces/robots/g1.py

def get_yaw(self):
    return self._pose_view.get_yaw()

has_bad_contacts

has_bad_contacts()

Source code in molmo_spaces/robots/g1.py

def has_bad_contacts(self):
    return self._pose_view.has_bad_contacts()

kinematics_wbc

kinematics_wbc(target_pos, target_rot=None, hand='right', avoid_self_collision=False, precision=False)

gold's G1Controller._solve_ik_wbc: mink IK on a standalone robot-only model synced from the live scene qpos. precision selects the tighter iteration/convergence settings gold used from PHASE_DESCEND on. Returns (arm, waist, ik_h, err).

Source code in molmo_spaces/robots/g1.py

def kinematics_wbc(
    self, target_pos, target_rot=None, hand="right", avoid_self_collision=False, precision=False
):
    """gold's G1Controller._solve_ik_wbc: mink IK on a standalone robot-only
    model synced from the live scene qpos. `precision` selects the tighter
    iteration/convergence settings gold used from PHASE_DESCEND on.
    Returns (arm, waist, ik_h, err).
    """
    self._ensure_wbc_ik_setup()

    assert self._wbc_hand_cfg is not None
    assert self._wbc_ik_cfg is not None
    assert self._wbc_scene_to_ik_qpos is not None
    assert self._wbc_posture_task is not None
    assert self._wbc_pelvis_task is not None
    assert self._wbc_limits is not None

    hcfg = self._wbc_hand_cfg[hand]

    ik_q = np.zeros(self._wbc_ik_cfg.model.nq, dtype=np.float64)
    for scene_qa, ik_qa, qsz in self._wbc_scene_to_ik_qpos:
        ik_q[ik_qa : ik_qa + qsz] = self.data.qpos[scene_qa : scene_qa + qsz]
    self._wbc_ik_cfg.update(ik_q)

    q_post = self._wbc_ik_cfg.q.copy()
    q_post[self._wbc_waist_qa] = 0.0
    self._wbc_posture_task.set_target(q_post)
    # Anchor pelvis xy to current (walking unaffected) but z to standing height
    # so the IK actively wants to stand back up when the wrist target allows it.
    pelvis_T = self._wbc_ik_cfg.get_transform_frame_to_world("pelvis", "body")
    pelvis_pos = pelvis_T.translation().copy()
    pelvis_pos[2] = HEIGHT_MAX
    self._wbc_pelvis_task.set_target(
        mink.SE3.from_rotation_and_translation(pelvis_T.rotation(), pelvis_pos)
    )
    rot = mink.SO3.from_matrix(target_rot) if target_rot is not None else mink.SO3.identity()
    hcfg["task"].set_target(
        mink.SE3.from_rotation_and_translation(rot, np.asarray(target_pos, dtype=np.float64))
    )

    max_iters = 60 if precision else 20
    conv_thresh = 0.0015 if precision else 0.005
    prev_err = float("inf")
    err = float("inf")
    limits = self._wbc_limits
    if avoid_self_collision and self._wbc_self_collision_limit is not None:
        limits = self._wbc_limits + [self._wbc_self_collision_limit]
    for step in range(max_iters):
        try:
            vel = mink.solve_ik(
                self._wbc_ik_cfg,
                [hcfg["task"], self._wbc_posture_task, self._wbc_pelvis_task],
                1e-2,
                "daqp",
                damping=1e-1,
                limits=limits,
            )
        except Exception:
            break
        vel *= hcfg["mask_h"]
        self._wbc_ik_cfg.integrate_inplace(vel, 1e-2)
        q_tmp = self._wbc_ik_cfg.q.copy()
        q_tmp[self._wbc_ik_fj_qa + 2] = np.clip(
            q_tmp[self._wbc_ik_fj_qa + 2], HEIGHT_MIN, HEIGHT_MAX
        )
        self._wbc_ik_cfg.update(q_tmp)
        err = float(np.linalg.norm(hcfg["task"].compute_error(self._wbc_ik_cfg)[:3]))
        if err < conv_thresh:
            break
        if step > 10 and err > prev_err - 1e-5:
            break
        prev_err = err
    q = self._wbc_ik_cfg.q
    arm = q[hcfg["arm_qa"]].astype(np.float32)
    waist = q[self._wbc_waist_qa].astype(np.float32)
    ik_h = float(np.clip(q[self._wbc_ik_fj_qa + 2], HEIGHT_MIN, HEIGHT_MAX))
    return arm, waist, ik_h, err

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

nav_action

nav_action(waypoint, height=None) -> ndarray

A full flat-15 action that walks toward waypoint while holding the current arm/gripper pose -- the navigation counterpart to the grasp policy's own action assembly (see g1_wbc.ACTION_DIM's layout: [vx, vy, yaw_rate, height, waist(3), right_arm(7), gripper]).

Source code in molmo_spaces/robots/g1.py

def nav_action(self, waypoint, height=None) -> np.ndarray:
    """A full flat-15 action that walks toward `waypoint` while holding the
    current arm/gripper pose -- the navigation counterpart to the grasp
    policy's own action assembly (see g1_wbc.ACTION_DIM's layout:
    [vx, vy, yaw_rate, height, waist(3), right_arm(7), gripper])."""
    action = np.zeros(15, dtype=np.float32)
    action[0:3] = self.waypoint_to_velocity_target(waypoint)
    action[3] = float(self._low_level._height_cmd if height is None else height)
    # waist stays 0 (level torso); arm/gripper hold their live pose so
    # navigating never disturbs whatever the arm was doing.
    q = self.data.qpos
    action[7:14] = q[self._qpos_ids[22:29]]
    action[14] = q[self._qpos_ids[29]]
    return action

pelvis_height

pelvis_height()

Source code in molmo_spaces/robots/g1.py

def pelvis_height(self):
    return self._pose_view.pelvis_height()

place

place(xy, yaw)

Source code in molmo_spaces/robots/g1.py

def place(self, xy, yaw):
    self.set_pose(np.asarray(xy, dtype=np.float64), float(yaw))
    self.set_defaults()
    self.zero_velocities()
    mj.mj_forward(self.model, self.data)

reset

reset() -> None

Robot ABC reset: DEFAULT_QPOS (the reference stack's settled standing pose, not G1Config.init_qpos) plus the WBC's internal state.

Source code in molmo_spaces/robots/g1.py

def reset(self) -> None:
    """Robot ABC reset: DEFAULT_QPOS (the reference stack's settled standing
    pose, not G1Config.init_qpos) plus the WBC's internal state."""
    self.set_defaults()
    reset_wbc = getattr(self._low_level, "_reset_wbc_state", None)
    if reset_wbc is not None:
        reset_wbc()

robot_model_root_name staticmethod

robot_model_root_name() -> str

Robot ABC hook used by the scene builder to find the robot's root body in its MJCF. Same body the deleted g1_old_reference used -- both wrapped the same g1_dex.xml.

Source code in molmo_spaces/robots/g1.py

@staticmethod
def robot_model_root_name() -> str:
    """Robot ABC hook used by the scene builder to find the robot's root
    body in its MJCF. Same body the deleted g1_old_reference used -- both wrapped the same
    g1_dex.xml."""
    return "pelvis"

sample_upper_pose

sample_upper_pose(np_random, radius=0.15)

Source code in molmo_spaces/robots/g1.py

def sample_upper_pose(self, np_random, radius=0.15):
    idx = self._UPPER_RAND_IDX
    # Waist gets 0.4x the arm's noise range so the trunk stays closer to neutral.
    per_dim = np.full(len(idx), radius, dtype=np.float64)
    per_dim[:3] = radius * 0.4
    sampled = DEFAULT_QPOS[idx] + np_random.uniform(-per_dim, per_dim)
    sampled[self._UPPER_GRIPPER_LOCAL] = float(np_random.uniform(GRIPPER_OPEN, GRIPPER_CLOSED))
    for i, jidx in enumerate(idx):
        jid = mj.mj_name2id(
            self.model, mj.mjtObj.mjOBJ_JOINT, f"{self._namespace}{JOINT_NAMES[jidx]}"
        )
        if jid >= 0:
            lo, hi = self.model.jnt_range[jid]
            sampled[i] = float(np.clip(sampled[i], lo, hi))
    return sampled

set_defaults

set_defaults()

Source code in molmo_spaces/robots/g1.py

def set_defaults(self):
    self.data.qpos[self._qpos_ids] = DEFAULT_QPOS
    valid = self.act_ids >= 0
    self.data.ctrl[self.act_ids[valid]] = DEFAULT_QPOS[valid]
    for i, name in enumerate(JOINT_NAMES):
        for prefix in ("walk_", "grasp_"):
            aid = mj.mj_name2id(
                self.model, mj.mjtObj.mjOBJ_ACTUATOR, f"{self._namespace}{prefix}{name}"
            )
            if aid >= 0:
                self.data.ctrl[aid] = DEFAULT_QPOS[i]
    if self.right_gripper_aid >= 0:
        self.data.ctrl[self.right_gripper_aid] = GRIPPER_OPEN
    for jname, qval in (
        ("right_Joint1_1", GRIPPER_OPEN),
        ("right_Joint2_1", GRIPPER_OPEN),
    ):
        jid = mj.mj_name2id(self.model, mj.mjtObj.mjOBJ_JOINT, f"{self._namespace}{jname}")
        if jid >= 0:
            self.data.qpos[self.model.jnt_qposadr[jid]] = qval

set_env

set_env(env)

Wires the low-level controller to env (mj model opt overrides, n_substeps, floor friction -- see g1_wbc.G1WholeBodyController. set_env). Called by env_g1ms.py's _load_scene once self is assigned to env.robot (set_env reads env.robot.n_substeps, so it can't run from inside init, before that assignment exists).

Source code in molmo_spaces/robots/g1.py

def set_env(self, env):
    """Wires the low-level controller to `env` (mj model opt overrides,
    n_substeps, floor friction -- see g1_wbc.G1WholeBodyController.
    set_env). Called by env_g1ms.py's _load_scene once `self` is
    assigned to `env.robot` (set_env reads env.robot.n_substeps, so it
    can't run from inside __init__, before that assignment exists)."""
    self._env = env
    self._low_level.set_env(env)

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_pose

set_pose(xy, yaw, z=STANDING_HEIGHT)

Source code in molmo_spaces/robots/g1.py

def set_pose(self, xy, yaw, z=STANDING_HEIGHT):
    self._pose_view.set_pose(xy, yaw, z=z)

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

solve_scene_ik

solve_scene_ik(pos, rot=None, hand='right', ik_joints=None, col_limit=None, use_height=True)

Full-scene mink IK (gold's GraspPolicy._solve_ik) on the live scene model. ik_joints and col_limit are the caller's, being grasp-specific. Returns {unprefixed_joint_name: qpos} for every joint.

Source code in molmo_spaces/robots/g1.py

def solve_scene_ik(
    self, pos, rot=None, hand="right", ik_joints=None, col_limit=None, use_height=True
):
    """Full-scene mink IK (gold's GraspPolicy._solve_ik) on the live scene
    model. `ik_joints` and `col_limit` are the caller's, being
    grasp-specific. Returns {unprefixed_joint_name: qpos} for every joint.
    """
    if ik_joints is None:
        ik_joints = set(HAND_ARM_JOINTS[hand]) | set(WAIST_JOINTS)
    site = HAND_SITE[hand]

    if self._ik_cfg is None:
        self._ik_cfg = mink.Configuration(self.model)
        self._ik_mdl = self._ik_cfg.model
        self._ik_fj_qa = self._ik_mdl.jnt_qposadr[
            mj.mj_name2id(
                self._ik_mdl, mj.mjtObj.mjOBJ_JOINT, self._namespace + "floating_base_joint"
            )
        ]
    assert self._ik_mdl, "Must have a valid MjModel for _ik_mdl by now"
    assert self._ik_fj_qa, "Must have a valid nparray for _ik_fj_qa by now"

    config = self._ik_cfg
    config.update(self.data.qpos.copy())

    mask = np.zeros(self._ik_mdl.nv)
    for jn in ik_joints:
        jid = mj.mj_name2id(self._ik_mdl, mj.mjtObj.mjOBJ_JOINT, self._namespace + jn)
        if jid >= 0:
            mask[self._ik_mdl.jnt_dofadr[jid]] = 1.0
    if use_height:
        fj_dof = self._ik_mdl.jnt_dofadr[
            mj.mj_name2id(
                self._ik_mdl, mj.mjtObj.mjOBJ_JOINT, self._namespace + "floating_base_joint"
            )
        ]
        mask[fj_dof + 2] = 1.0

    ht = mink.FrameTask(
        frame_name=self._namespace + site,
        frame_type="site",
        position_cost=100,
        orientation_cost=1,
        lm_damping=1,
    )
    post = mink.PostureTask(self._ik_mdl, cost=1e-2)
    post.set_target_from_configuration(config)
    r = mink.SO3.from_matrix(rot) if rot is not None else mink.SO3.identity()
    ht.set_target(mink.SE3.from_rotation_and_translation(r, np.asarray(pos, dtype=np.float64)))

    limits = [mink.ConfigurationLimit(self._ik_mdl)]
    if col_limit is not None:
        limits.append(col_limit)
    prev = float("inf")
    for step in range(300):
        try:
            vel = mink.solve_ik(config, [ht, post], IK_DT, "daqp", damping=1e-1, limits=limits)
        except Exception:
            break
        vel *= mask
        config.integrate_inplace(vel, IK_DT)
        q = config.q.copy()
        q[self._ik_fj_qa + 2] = np.clip(q[self._ik_fj_qa + 2], HEIGHT_MIN, HEIGHT_MAX)
        config.update(q)
        err = np.linalg.norm(ht.compute_error(config)[:3])
        if err < 0.001:
            break
        if step == 100 and err > 0.1:
            break
        if step > 100 and err > prev - 1e-5:
            break
        prev = err

    if use_height:
        ik_h = np.clip(config.q[self._ik_fj_qa + 2], HEIGHT_MIN, HEIGHT_MAX)
        self.model.qpos_spring[self._fj_scene + 2] = ik_h
        self.model.dof_damping[self.model.jnt_dofadr[self._freejoint_id] + 2] = (
            IK_HEIGHT_DAMPING
        )

    out = {}
    for jn in self._stj:
        if jn == self._namespace + "floating_base_joint":
            continue
        jid = mj.mj_name2id(self._ik_mdl, mj.mjtObj.mjOBJ_JOINT, jn)
        if jid < 0:
            continue
        key = jn[len(self._namespace) :] if jn.startswith(self._namespace) else jn
        out[key] = config.q[self._ik_mdl.jnt_qposadr[jid]]
    return out

state_is_finite

state_is_finite()

Source code in molmo_spaces/robots/g1.py

def state_is_finite(self):
    d = self.data
    return np.isfinite(d.qpos).all() and np.isfinite(d.qvel).all() and np.isfinite(d.ctrl).all()

update_control

update_control(action_command_dict) -> None

Robot.update_control: a move-group dict, whose "legs_waist" entry is [vx, vy, yaw_rate, height, waist(3)], translated to the reference controller's (cmd3, height, waist3). execute_action is the reference stack's own flat-15 path and is untouched.

Source code in molmo_spaces/robots/g1.py

def update_control(self, action_command_dict) -> None:
    """Robot.update_control: a move-group dict, whose "legs_waist" entry is
    [vx, vy, yaw_rate, height, waist(3)], translated to the reference
    controller's (cmd3, height, waist3). `execute_action` is the reference
    stack's own flat-15 path and is untouched.
    """
    # The pelvis has no actuators, so "base" (an absolute [x, y, theta]
    # waypoint) and "base_velocity" ([vx, vy, yaw_rate]) both become a
    # legs_waist command here.
    action_command_dict = dict(action_command_dict)
    waypoint = action_command_dict.pop("base", None)
    base_velocity = action_command_dict.pop("base_velocity", None)
    if waypoint is not None and len(waypoint) == 3:
        action_command_dict["legs_waist"] = self._legs_waist_target(
            self.waypoint_to_velocity_target(waypoint)
        )
    elif base_velocity is not None and len(base_velocity) >= 3:
        bv = np.asarray(base_velocity, dtype=np.float32)
        # Same floored clip the waypoint path applies: a nav policy's
        # smoothstep brake otherwise spends real time in
        # [VELOCITY_DEADBAND, MIN_LINEAR_VEL), the regime the WBC's
        # stand/walk switch gets stuck in.
        action_command_dict["legs_waist"] = self._legs_waist_target(
            np.array(
                [
                    self._floored_clip(
                        bv[0], MIN_LINEAR_VEL, MAX_LINEAR_VEL, VELOCITY_DEADBAND
                    ),
                    self._floored_clip(
                        bv[1], MIN_LINEAR_VEL, MAX_LINEAR_VEL, VELOCITY_DEADBAND
                    ),
                    self._floored_clip(bv[2], MIN_YAW_RATE, MAX_YAW_RATE, VELOCITY_DEADBAND),
                ],
                dtype=np.float32,
            )
        )

    controllers = self.controllers
    for mg_id, target in action_command_dict.items():
        controller = controllers.get(mg_id)
        if controller is None or target is None:
            continue
        target = np.asarray(target, dtype=np.float32)
        if mg_id == "legs_waist":
            if target.shape[0] != NUM_LEGS_WAIST_TARGET_DIMS:
                raise ValueError(
                    f"legs_waist target must be {NUM_LEGS_WAIST_TARGET_DIMS} values "
                    f"[vx, vy, yaw_rate, height, waist(3)], got {target.shape[0]}"
                )
            controller.set_target((target[0:3], float(target[3]), target[4:7]))
        else:
            controller.set_target(target if target.size > 1 else float(target[0]))

waypoint_to_velocity_target

waypoint_to_velocity_target(waypoint) -> ndarray

Absolute [x, y, yaw] waypoint -> the WBC's [vx, vy, yaw_rate] base command, in the robot's own body frame. Same bridge (and same constants) as native molmo_spaces/robots/g1.py, which is what lets a molmo_spaces navigation policy drive this robot: the reference stack has no such bridge because its policy emits base velocities directly.

Source code in molmo_spaces/robots/g1.py

def waypoint_to_velocity_target(self, waypoint) -> np.ndarray:
    """Absolute [x, y, yaw] waypoint -> the WBC's [vx, vy, yaw_rate] base
    command, in the robot's own body frame. Same bridge (and same
    constants) as native molmo_spaces/robots/g1.py, which is what lets a
    molmo_spaces navigation policy drive this robot: the reference stack
    has no such bridge because its policy emits base velocities directly.
    """
    xy = self._pose_view.get_xy()
    yaw = self._pose_view.get_yaw()

    dx, dy = waypoint[0] - xy[0], waypoint[1] - xy[1]
    local_vx = np.cos(yaw) * dx + np.sin(yaw) * dy
    local_vy = -np.sin(yaw) * dx + np.cos(yaw) * dy
    yaw_error = normalize_ang_error(waypoint[2] - yaw)

    yaw_rate = self._floored_clip(yaw_error, MIN_YAW_RATE, MAX_YAW_RATE, VELOCITY_DEADBAND)
    if abs(yaw_error) > YAW_GATE_THRESHOLD:
        # Turn-then-drive: while heading is substantially off, correcting it
        # and translating at once fights itself (the gait's own turning drift
        # keeps re-triggering a position correction that never converges).
        vx = vy = 0.0
    else:
        vx = self._floored_clip(local_vx, MIN_LINEAR_VEL, MAX_LINEAR_VEL, VELOCITY_DEADBAND)
        vy = self._floored_clip(local_vy, MIN_LINEAR_VEL, MAX_LINEAR_VEL, VELOCITY_DEADBAND)
    return np.array([vx, vy, yaw_rate], dtype=np.float32)

zero_velocities

zero_velocities()

Source code in molmo_spaces/robots/g1.py

def zero_velocities(self):
    self._pose_view.zero_velocities()

G1RobotView

G1RobotView(model, data, body_id, freejoint_id, cam_ids, dof_ids, namespace: str = PREFIX)

Pose/contact/visibility helpers for the G1 (the reference stack's view); G1Robot keeps same-named pass-throughs.

Methods:

Name	Description
check_object_visibility
close	Free the lazily-created visibility renderer (and its GL/EGL context).
get_xy
get_yaw
has_bad_contacts
pelvis_height
set_pose
zero_velocities

Attributes:

Name	Type	Description
data
model

Source code in molmo_spaces/robots/g1.py

def __init__(
    self, model, data, body_id, freejoint_id, cam_ids, dof_ids, namespace: str = PREFIX
):
    self.model = model
    self.data = data
    self._body_id = body_id
    self._freejoint_id = freejoint_id
    self._cam_ids = cam_ids
    self._dof_ids = dof_ids
    self._namespace = namespace
    self._renderer = None

data instance-attribute

data = data

model instance-attribute

model = model

check_object_visibility

check_object_visibility(body_id, threshold=2e-05)

Source code in molmo_spaces/robots/g1.py

def check_object_visibility(self, body_id, threshold=0.00002):
    # threshold = minimum fraction of the 224x224 segmentation frame the
    # object must occupy in at least one camera. ~0.00002 (~1px) restores
    # the original presence check: a hard 0.2% (~100px) gate is unsatisfiable
    # for small/distant objects at far nav spawns, so placement fails after
    # its retry budget and crashes rollout workers. Keep this near 1px.
    if not self._cam_ids:
        return True
    if self._renderer is None:
        # procthor scenes can exceed the default max_geom.
        self._renderer = mj.Renderer(
            self.model, 224, 224, max_geom=max(20000, self.model.ngeom * 4)
        )
    m = self.model
    geom_ids = set()
    stack = [body_id]
    while stack:
        bid = stack.pop()
        for gid in range(m.ngeom):
            if m.geom_bodyid[gid] == bid:
                geom_ids.add(gid)
        for cbid in range(m.nbody):
            if m.body_parentid[cbid] == bid and cbid != bid:
                stack.append(cbid)

    for cam_id in self._cam_ids:
        try:
            self._renderer.update_scene(self.data, cam_id)
            self._renderer.enable_segmentation_rendering()
            seg = self._renderer.render()
            self._renderer.disable_segmentation_rendering()
        except IndexError:
            return True  # segid overflow — treat as visible to avoid crashing reset.
        seg0 = seg[:, :, 0]
        vis = np.isin(seg0, list(geom_ids)).sum()  # object's visible pixels
        if vis / seg0.size >= threshold:  # fraction of the frame
            return True
    return False

close

close()

Free the lazily-created visibility renderer (and its GL/EGL context). Must be called before dropping the robot on scene reload — otherwise the renderer's framebuffer leaks on the render GPU because del-based EGL teardown is unreliable, so VRAM creeps to OOM across reloads.

Source code in molmo_spaces/robots/g1.py

def close(self):
    """Free the lazily-created visibility renderer (and its GL/EGL context).
    Must be called before dropping the robot on scene reload — otherwise the
    renderer's framebuffer leaks on the render GPU because __del__-based EGL
    teardown is unreliable, so VRAM creeps to OOM across reloads."""
    r = getattr(self, "_renderer", None)
    if r is not None:
        with contextlib.suppress(Exception):
            r.close()
        self._renderer = None

get_xy

get_xy()

Source code in molmo_spaces/robots/g1.py

def get_xy(self):
    return self.data.xpos[self._body_id, :2].copy()

get_yaw

get_yaw()

Source code in molmo_spaces/robots/g1.py

def get_yaw(self):
    quat = self.data.xquat[self._body_id]
    return _R.from_quat(quat[[1, 2, 3, 0]]).as_euler("xyz")[2]

has_bad_contacts

has_bad_contacts()

Source code in molmo_spaces/robots/g1.py

def has_bad_contacts(self):
    m, d = self.model, self.data
    for i in range(d.ncon):
        con = d.contact[i]
        g1, g2 = int(con.geom1), int(con.geom2)
        n1 = mj.mj_id2name(m, mj.mjtObj.mjOBJ_BODY, m.geom_bodyid[g1]) or ""
        n2 = mj.mj_id2name(m, mj.mjtObj.mjOBJ_BODY, m.geom_bodyid[g2]) or ""
        r1, r2 = n1.startswith(self._namespace), n2.startswith(self._namespace)
        if r1 == r2:
            continue
        scene_geom = g2 if r1 else g1
        if _is_floor_geom(mj.mj_id2name(m, mj.mjtObj.mjOBJ_GEOM, scene_geom) or ""):
            continue
        return True
    return False

pelvis_height

pelvis_height()

Source code in molmo_spaces/robots/g1.py

def pelvis_height(self):
    qadr = self.model.jnt_qposadr[self._freejoint_id]
    return float(self.data.qpos[qadr + 2])

set_pose

set_pose(xy, yaw, z=STANDING_HEIGHT)

Source code in molmo_spaces/robots/g1.py

def set_pose(self, xy, yaw, z=STANDING_HEIGHT):
    qadr = self.model.jnt_qposadr[self._freejoint_id]
    c, s = np.cos(yaw), np.sin(yaw)
    px = xy[0] - c * PELVIS_FORWARD_OFFSET
    py = xy[1] - s * PELVIS_FORWARD_OFFSET
    self.data.qpos[qadr : qadr + 3] = [px, py, z]
    self.data.qpos[qadr + 3 : qadr + 7] = _R.from_euler("z", yaw).as_quat(scalar_first=True)

zero_velocities

zero_velocities()

Source code in molmo_spaces/robots/g1.py

def zero_velocities(self):
    dadr = self.model.jnt_dofadr[self._freejoint_id]
    self.data.qvel[dadr : dadr + 6] = 0.0
    for i in range(len(JOINT_NAMES)):
        self.data.qvel[self._dof_ids[i]] = 0.0

i2rt_yam

Classes:

Name	Description
I2rtYamRobot

I2rtYamRobot

I2rtYamRobot(mj_data: MjData, config: BaseRobotConfig)

Bases: Robot

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	YAM has a single arm move group.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers		One or more controllers for the robot joints / wheels / etc.
kinematics		kinematic solver for the robot
mj_data
mj_model
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		parallel kinematic solver for the robot
robot_config
robot_view		robot view for interfacing with joints / links / actuators

Source code in molmo_spaces/robots/i2rt_yam.py

def __init__(self, mj_data: mj.MjData, config: BaseRobotConfig) -> None:
    super().__init__(mj_data, config)

    assert config.robot_view_factory, (
        "Something went wrong, 'robot_view_factory' shouldn't be None"
    )
    self._robot_view = config.robot_view_factory(mj_data, config.robot_namespace)
    self._kinematics = MlSpacesKinematics(config)

    self._parallel_kinematics = SimpleWarpKinematics(config)

    arm_controller_cls = (
        JointPosController
        if config.command_mode == {} or config.command_mode.get("arm") == "joint_position"
        else JointRelPosController
    )
    self._controllers: dict[str, Controller] = {
        "arm": arm_controller_cls(self._robot_view.get_move_group("arm")),
        "gripper": JointPosController(self._robot_view.get_move_group("gripper")),
    }

controllers property

controllers

One or more controllers for the robot joints / wheels / etc.

kinematics property

kinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

robot view for interfacing with joints / links / actuators

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: I2rtYamRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/i2rt_yam.py

@classmethod
def add_robot_to_scene(  # pyright: ignore[reportIncompatibleMethodOverride] # ty: ignore[invalid-method-override]
    cls,
    robot_config: I2rtYamRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    # TODO(wilbert): this part is iffy, we shouldn't expect to receive either both vec3 and vec2
    # here. The caller should just send a vec3 and if he just wants xy then the last element
    # should be just 0. We could maybe change list[float] to tuple[float,float,float] and so on
    pos_xyz = pos + [0.0] if len(pos) == 2 else pos.copy()

    # Create a mocap body to control the robot base pose
    robot_body = spec.worldbody.add_body(
        name=f"{prefix}base",
        pos=pos_xyz,
        quat=quat,
        mocap=True,
    )

    if robot_config.base_size is not None:
        base_height = robot_config.base_size[2]

        # Create a base material (plain dark wood color)
        material_name = f"{prefix}robot_base_material"
        spec.add_material(name=material_name, rgba=[0.3, 0.2, 0.1, 1.0])

        # Add base geometry (platform)
        robot_body.add_geom(
            type=mj.mjtGeom.mjGEOM_BOX,
            size=[x / 2 for x in robot_config.base_size],
            pos=[0, 0, base_height / 2],
            material=material_name,
            group=0,  # Visual group
        )
        attach_frame = robot_body.add_frame(pos=[0, 0, base_height])
    else:
        attach_frame = robot_body.add_frame()

    robot_spec = cls._load_robot_spec(robot_config, strip_meshes=strip_meshes)
    robot_root_name = cls.robot_model_root_name()
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: BaseRobotConfig):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    # TODO: control overrides will break if non-robot actuators are present
    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

    force_limit = robot_config.force_limit
    if force_limit is not None:
        log.debug(f"Applying force limits to robot {cls.robot_model_root_name()}")
        assert len(force_limit) <= len(actuators), (
            "number of force limits cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(force_limit)]):
            actuator.forcelimited = 1
            actuator.forcerange[:] = [-force_limit[i], force_limit[i]]

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/abstract.py

def compute_control(self) -> None:
    """
    Compute and set the control inputs to the robot actuators based on the
    current state and the targets set by the user.
    Must be called after update_control().
    """
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

create_robot_sensors

create_robot_sensors()

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/i2rt_yam.py

def create_robot_sensors(self):
    return super().create_robot_sensors() + [
        TCPPoseSensor(uuid="tcp_pose"),
    ]

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

YAM has a single arm move group.

Source code in molmo_spaces/robots/i2rt_yam.py

def get_arm_move_group_ids(self) -> list[str]:
    """YAM has a single arm move group."""
    return ["arm"]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset() -> None

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/i2rt_yam.py

def reset(self) -> None:
    for mg_id, default_pos in self.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = np.array(default_pos)

robot_model_root_name staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/i2rt_yam.py

@staticmethod
def robot_model_root_name() -> str:
    return "arm"

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, ndarray]) -> None

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/abstract.py

def update_control(self, action_command_dict: dict[str, np.ndarray]) -> None:
    """
    Update the control targets to the robot based on the provided action commands.
    Does not set the control inputs to the robot actuators. See compute_control().

    Args:
        action_command_dict: Dictionary containing action commands for the robot
                             based on the move groups ids to be used.
    """
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

mobile_franka

Classes:

Name	Description
MobileFrankaRobot

Attributes:

Name	Type	Description
data
house_xml_path
houses
init_qpos
log
mg
model
robot_config
spec
view

data module-attribute

data = mj.MjData(model)

house_xml_path module-attribute

house_xml_path = houses['val'][0]['base']

houses module-attribute

houses = get_procthor_10k_houses(split='val')

init_qpos module-attribute

init_qpos = {key: np.array(val) for key, val in robot_config.init_qpos.items()}

log module-attribute

log = logging.getLogger(__name__)

mg module-attribute

mg = view.get_move_group(mg_id)

model module-attribute

model = spec.compile()

robot_config module-attribute

robot_config = MobileFrankaRobotConfig(base_size=[0.5, 0.5, 0.75])

spec module-attribute

spec = mj.MjSpec.from_file(house_xml_path)

view module-attribute

view = robot_config.robot_view_factory(data, robot_config.robot_namespace)

MobileFrankaRobot

MobileFrankaRobot(mj_data: MjData, config: BaseRobotConfig)

Bases: Robot

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_base_material
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	Get the move group IDs that should have TCP-bounded noise applied.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state or a provided state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
controllers	dict[str, Controller]	One or more controllers for the robot joints / wheels / etc.
kinematics		kinematic solver for the robot
mj_data
mj_model
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		parallel kinematic solver for the robot
robot_config
robot_view		robot view for interfacing with joints / links / actuators

Source code in molmo_spaces/robots/mobile_franka.py

def __init__(
    self,
    mj_data: mj.MjData,
    config: BaseRobotConfig,
) -> None:
    super().__init__(mj_data, config)

    assert config.robot_view_factory, (
        "Something went wrong, 'robot_view_factory' shouldn't be None"
    )
    self._robot_view = config.robot_view_factory(mj_data, config.robot_namespace)
    self._kinematics = MlSpacesKinematics(config)

    self._parallel_kinematics = SimpleWarpKinematics(config)
    arm_controller_cls = (
        JointPosController
        if config.command_mode == {} or config.command_mode["arm"] == "joint_position"
        else JointRelPosController
    )

    BASE_CONTROLLER_CLS_OPTIONS = {
        "holo_joint_planar_position": JointPosController,
        "holo_joint_rel_planar_position": JointRelPosController,
    }

    cmd_mode = config.command_mode["base"]
    assert cmd_mode and cmd_mode in BASE_CONTROLLER_CLS_OPTIONS, (
        f"Must pass a valid command mode, but got {cmd_mode}"
    )
    base_controller_cls = BASE_CONTROLLER_CLS_OPTIONS[cmd_mode]

    self._controllers: dict[str, Controller] = {
        "base": base_controller_cls(self._robot_view.get_move_group("base")),
        "arm": arm_controller_cls(self._robot_view.get_move_group("arm")),
        "gripper": JointPosController(self._robot_view.get_move_group("gripper")),
    }

controllers property

controllers: dict[str, Controller]

One or more controllers for the robot joints / wheels / etc.

kinematics property

kinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

robot view for interfacing with joints / links / actuators

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: MobileFrankaRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/mobile_franka.py

@classmethod
def add_robot_to_scene(  # pyright: ignore[reportIncompatibleMethodOverride] # ty: ignore[invalid-method-override]
    cls,
    robot_config: MobileFrankaRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    def add_slider_act(
        name: str, ctrlrange: float, gainprm: float, biasprm: list[float], gear_idx: int
    ):
        act = spec.add_actuator(
            name=f"{prefix}{name}",
            target=f"{prefix}base_site",
            refsite=f"{prefix}world",
            ctrlrange=[-ctrlrange, ctrlrange],
            trntype=mj.mjtTrn.mjTRN_SITE,
            biastype=mj.mjtBias.mjBIAS_AFFINE,
            gear=[1 if i == gear_idx else 0 for i in range(6)],
        )
        act.gainprm[0] = gainprm
        act.biasprm[: len(biasprm)] = biasprm
        return act

    pos = pos + [0.005] if len(pos) == 2 else pos

    material_name = cls.create_robot_base_material(
        robot_config, spec, prefix, randomize_textures
    )

    robot_body = spec.worldbody.add_body(
        name=f"{prefix}base",
        pos=pos,
        quat=quat,
    )
    robot_body.add_site(name=f"{prefix}base_site", pos=[0, 0, 0], quat=[1, 0, 0, 0])
    base_height = robot_config.base_size[2]

    init_rot = R.from_quat(quat, scalar_first=True)
    init_rpy = init_rot.as_euler("xyz")
    assert np.allclose(init_rpy[:2], [0, 0]), (
        f"Initial roll and pitch are not zero: {init_rpy[:2]}"
    )
    init_yaw = init_rpy[2]

    # Add base geometry (wooden platform)
    robot_body.add_geom(
        type=mj.mjtGeom.mjGEOM_BOX,
        size=[x / 2 for x in robot_config.base_size],
        pos=[0, 0, base_height / 2],
        material=material_name,
        group=0,  # Visual group
    )
    attach_frame = robot_body.add_frame(pos=[0, 0, base_height])

    robot_spec = cls._load_robot_spec(robot_config, strip_meshes=strip_meshes)
    robot_root_name = cls.robot_model_root_name()
    robot_root = robot_spec.body(robot_root_name)
    if robot_root is None:
        raise ValueError(f"Robot {robot_root_name=} not found in {robot_spec}")
    attach_frame.attach_body(robot_root, prefix, "")

    spec.worldbody.add_site(name=f"{prefix}world", pos=[0, 0, 0.005], quat=[1, 0, 0, 0])

    for gear_idx, jnt_name in enumerate(["base_x", "base_y"]):
        act_name = jnt_name + "_act"
        params = robot_config.base_control_params[act_name]
        jnt_axis = np.zeros(3)
        jnt_axis[gear_idx] = 1
        jnt_axis = init_rot.inv().apply(jnt_axis)
        robot_body.add_joint(
            type=mj.mjtJoint.mjJNT_SLIDE,
            name=f"{prefix}{jnt_name}",
            axis=jnt_axis,
            range=[-params["ctrlrange"], params["ctrlrange"]],
            ref=pos[gear_idx],
        )
        # use damping ratio if available, otherwise use kd (which should be negative in biasprm)
        add_slider_act(
            act_name,
            params["ctrlrange"],
            params["kp"],
            [0, -params["kp"], params.get("damping_ratio", -params.get("kd", 0.0))],
            gear_idx,
        )

    theta_act_params = robot_config.base_control_params["base_theta_act"]
    robot_body.add_joint(
        type=mj.mjtJoint.mjJNT_HINGE,
        name=f"{prefix}base_theta",
        axis=[0, 0, 1],
        ref=init_yaw,
    )
    theta_act = spec.add_actuator(
        name=f"{prefix}base_theta_act",
        target=f"{prefix}base_theta",
        trntype=mj.mjtTrn.mjTRN_JOINT,
        biastype=mj.mjtBias.mjBIAS_AFFINE,
    )
    theta_act.gainprm[0] = theta_act_params["kp"]
    theta_act.biasprm[1] = -theta_act_params["kp"]
    # use damping ratio if available, otherwise use kd (which should be negative in biasprm)
    theta_act.biasprm[2] = theta_act_params.get(
        "damping_ratio", -theta_act_params.get("kd", 0.0)
    )

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Each robot subclass can override this to customize noise behavior. Default implementation applies TCP-bounded noise to arm move groups returned by get_arm_move_group_ids().

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/abstract.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Each robot subclass can override this to customize noise behavior.
    Default implementation applies TCP-bounded noise to arm move groups
    returned by get_arm_move_group_ids().

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    noise_config = self.robot_config.action_noise_config
    assert noise_config, "Something is wrong here, 'noise_config' shouldn't be None"

    if not noise_config.enabled:
        return action

    arm_mg_ids = self.get_arm_move_group_ids()
    if not arm_mg_ids:
        return action

    noisy_action = dict(action)  # shallow copy

    for mg_id in arm_mg_ids:
        if mg_id not in action or action[mg_id] is None:
            continue

        commanded_joint_pos = np.asarray(action[mg_id])
        noisy_joint_pos = self._apply_tcp_noise_to_move_group(
            mg_id, commanded_joint_pos, noise_config
        )
        noisy_action[mg_id] = noisy_joint_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/abstract.py

@classmethod
def apply_control_overrides(cls, spec: MjSpec, robot_config: BaseRobotConfig):
    if robot_config.gravcomp:
        log.debug(f"Applying gravity compensation to robot {robot_config.name}")
        body_name = robot_config.robot_namespace + cls.robot_model_root_name()
        robot_body = spec.body(body_name)
        assert robot_body is not None, f"Robot body {body_name} not found in spec"
        for body in robot_body.find_all("body"):
            body: mujoco.MjsBody
            body.gravcomp = 1.0

    # TODO: control overrides will break if non-robot actuators are present
    stiffness, damping = robot_config.K_stiffness, robot_config.K_damping
    if stiffness is not None and damping is not None:
        assert len(stiffness) == len(damping), (
            "K_stiffness and K_damping must have the same length"
        )

    actuators: list[mujoco.MjsActuator] = spec.actuators
    if stiffness is not None:
        log.debug(f"Applying stiffness gains to robot {cls.robot_model_root_name()}")
        assert len(stiffness) <= len(actuators), (
            "number of stiffness gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(stiffness)]):
            actuator.gainprm[0] = stiffness[i]
            actuator.biasprm[1] = -stiffness[i]
    if damping is not None:
        log.debug(f"Applying damping gains to robot {cls.robot_model_root_name()}")
        assert len(damping) <= len(actuators), (
            "number of damping gains cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(damping)]):
            actuator.biasprm[2] = -damping[i]

    force_limit = robot_config.force_limit
    if force_limit is not None:
        log.debug(f"Applying force limits to robot {cls.robot_model_root_name()}")
        assert len(force_limit) <= len(actuators), (
            "number of force limits cannot exceed number of actuators"
        )
        for i, actuator in enumerate(actuators[: len(force_limit)]):
            actuator.forcelimited = 1
            actuator.forcerange[:] = [-force_limit[i], force_limit[i]]

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/abstract.py

def compute_control(self) -> None:
    """
    Compute and set the control inputs to the robot actuators based on the
    current state and the targets set by the user.
    Must be called after update_control().
    """
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

create_robot_base_material classmethod

create_robot_base_material(robot_config: MobileFrankaRobotConfig, spec: MjSpec, prefix: str, randomize_base_texture: bool) -> str

Source code in molmo_spaces/robots/mobile_franka.py

@classmethod
def create_robot_base_material(
    cls,
    robot_config: MobileFrankaRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    randomize_base_texture: bool,
) -> str:  # LOL(wilbert): lololol, why were we returning None when we actually return str xD
    texture_dir = robot_config.get_robot_dir() / "assets" / "base_textures"
    assert texture_dir.is_dir(), f"Texture directory {texture_dir} does not exist"
    texture_path: Path | None = None
    if randomize_base_texture:
        texture_paths = list(texture_dir.glob("*.png"))
        texture_paths.sort(key=lambda x: x.name)
        assert len(texture_paths) > 0, f"No robot base texture paths found in {texture_dir}"
        log.debug(f"Found {len(texture_paths)} robot base texture paths")
        texture_path = random.choice(texture_paths)
    else:
        texture_path = texture_dir / "DarkWood2.png"
        assert texture_path.is_file(), f"Default texture {texture_path} does not exist"

    texture_name = f"{prefix}robot_base_texture"
    spec.add_texture(
        name=texture_name,
        type=mj.mjtTexture.mjTEXTURE_CUBE,
        file=str(texture_path),
    )
    log.debug(f"Successfully created texture from {texture_path}")

    material_name = f"{prefix}robot_base_material"
    robot_base_mat = spec.add_material(name=material_name)
    robot_base_mat.textures[mj.mjtTextureRole.mjTEXROLE_RGB] = texture_name
    log.debug(f"Successfully created material {material_name}")
    return material_name

create_robot_sensors

create_robot_sensors()

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/mobile_franka.py

def create_robot_sensors(self):
    return super().create_robot_sensors() + [
        TCPPoseSensor(uuid="tcp_pose"),
    ]

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

Get the move group IDs that should have TCP-bounded noise applied.

Override in subclass to specify which move groups are arms. Default returns empty list (no noise applied).

Source code in molmo_spaces/robots/mobile_franka.py

def get_arm_move_group_ids(self) -> list[str]:
    return ["arm"]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset() -> None

Reset the robot to its initial state or a provided state.

Source code in molmo_spaces/robots/mobile_franka.py

def reset(self) -> None:
    for mg_id, default_pos in self.robot_config.init_qpos.items():
        if mg_id in self._robot_view.move_group_ids():
            self._robot_view.get_move_group(mg_id).joint_pos = np.array(default_pos)

robot_model_root_name staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/mobile_franka.py

@staticmethod
def robot_model_root_name() -> str:
    return "fr3_link0"

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, ndarray]) -> None

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/abstract.py

def update_control(self, action_command_dict: dict[str, np.ndarray]) -> None:
    """
    Update the control targets to the robot based on the provided action commands.
    Does not set the control inputs to the robot actuators. See compute_control().

    Args:
        action_command_dict: Dictionary containing action commands for the robot
                             based on the move groups ids to be used.
    """
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

rby1

Classes:

Name	Description
RBY1	RBY1 Robot class for the RBY1 robot.

RBY1

RBY1(mj_data: MjData, config: BaseRobotConfig)

Bases: Robot

RBY1 Robot class for the RBY1 robot. This class extends the Robot class and provides specific implementations for the RBY1 robot including the robot view, controllers, and a kinematic solver.

The mode in which the robot arm, gripper, or base is commanded (Eg. ee position command or joint position command) can be set using the arm_command_mode, gripper_command_mode, and base_command_mode parameters in the robot configuration.

The move_group i.e., the set of robot parts that are to be moved can be dynamically changed, e.g., move base for navigation, then move both base and arm for opening doors.

Methods:

Name	Description
add_robot_to_scene	Add a robot to a scene, taking care of any robot-specific considerations.
apply_action_noise	Apply action noise to the commanded action.
apply_control_overrides
compute_control	Compute and set the control inputs to the robot actuators based on the
create_robot_sensors	Get robot-specific sensors that should be registerd with a Task's sensor suite.
get_arm_move_group_ids	RBY1 has two independent arms - each gets independent noise.
get_joint_position	Get the current joint positions of the move groups
get_joint_ranges	Get the joint ranges of the move groups
get_world_pose_tf_mat	Get the robot's world pose transformation matrix.
last_unnoised_cmd_joint_pos	Get the last unnoised joint position commands for the robot.
reset	Reset the robot to its initial state.
robot_model_root_name	The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.
set_joint_pos	Set all the robot's joint positions to the specified values.
set_stationary	Set all controllers of the robot to stationary mode (if not already set),
set_world_pose	Set the robot's world pose to the specified pose in the world frame.
update_control	Update the control targets to the robot based on the provided action commands.

Attributes:

Name	Type	Description
arm_command_mode
arm_command_modes
base_command_mode
base_command_modes
controllers	dict[str, Controller]	One or more controllers for the robot joints / wheels / etc.
gripper_command_mode
gripper_command_modes
head_command_mode
kinematics		kinematic solver for the robot
mj_data
mj_model
namespace		robot namespace used to differentiate between one or multiple robots and the environment
parallel_kinematics		parallel kinematic solver for the robot
robot_config
robot_view		robot view for interfacing with joints / links / actuators
torso_command_mode
torso_command_modes

Source code in molmo_spaces/robots/rby1.py

def __init__(self, mj_data: mj.MjData, config: BaseRobotConfig) -> None:
    super().__init__(mj_data, config)

    assert config.robot_view_factory, (
        "Something went wrong, 'robot_view_factory' shouldn't be None"
    )

    self._namespace = self.robot_config.robot_namespace

    # TODO(wilbert): uhmmm, make sure we change the design of these configs and views to not
    # have to be forced to use getattr
    self._use_holo_base = getattr(self.robot_config, "use_holo_base")  # noqa: B009

    self._robot_view = RBY1RobotView(mj_data, self.namespace, holo_base=self._use_holo_base)
    self._kinematics = MlSpacesKinematics(self.robot_config)

    # Create controllers:

    # All the joint actuators are position controlled.
    # However, they can be commanded in other modes like velocity, ee position, etc. and the controller will
    # perform the necessary conversions to joint position.
    self.arm_command_modes = [
        "joint_position",
        "joint_rel_position",
        "joint_velocity",
        "ee_position",
        "ee_velocity",
    ]
    if self.robot_config.command_mode["arm"] is not None:
        assert self.robot_config.command_mode["arm"] in self.arm_command_modes, (
            f"Arm command mode {self.robot_config.command_mode['arm']} not in {self.arm_command_modes}"
        )
    self.arm_command_mode = (
        "joint_position"
        if not self.robot_config.command_mode["arm"]
        else self.robot_config.command_mode["arm"]
    )
    if self.arm_command_mode == "joint_rel_position":
        left_arm_controller = JointRelPosController(self.robot_view.get_move_group("left_arm"))
        right_arm_controller = JointRelPosController(
            self.robot_view.get_move_group("right_arm")
        )
    elif self.arm_command_mode == "joint_position":
        left_arm_controller = JointPosController(self.robot_view.get_move_group("left_arm"))
        right_arm_controller = JointPosController(self.robot_view.get_move_group("right_arm"))
    else:
        raise NotImplementedError(
            f"Arm command mode {self.arm_command_mode} not implemented yet."
        )

    # Gripper command modes - separate from arm command modes
    self.gripper_command_modes = [
        "joint_position",
        "joint_rel_position",
        "joint_velocity",
    ]
    if self.robot_config.command_mode["gripper"] is not None:
        assert self.robot_config.command_mode["gripper"] in self.gripper_command_modes, (
            f"Gripper command mode {self.robot_config.command_mode['gripper']} not in {self.gripper_command_modes}"
        )
    self.gripper_command_mode = (
        "joint_position"
        if not self.robot_config.command_mode["gripper"]
        else self.robot_config.command_mode["gripper"]
    )
    if self.gripper_command_mode == "joint_rel_position":
        left_gripper_controller = JointRelPosController(
            self.robot_view.get_move_group("left_gripper")
        )
        right_gripper_controller = JointRelPosController(
            self.robot_view.get_move_group("right_gripper")
        )
    elif self.gripper_command_mode == "joint_position":
        left_gripper_controller = JointPosController(
            self.robot_view.get_move_group("left_gripper")
        )
        right_gripper_controller = JointPosController(
            self.robot_view.get_move_group("right_gripper")
        )
    else:
        raise NotImplementedError(
            f"Gripper command mode {self.gripper_command_mode} not implemented yet."
        )

    # The base actuators (wheels) are usually velocity controlled. If using virtual holonomic joints, they are position controlled.
    # Base actuators can still be commanded in other modes like planar position, planar velocity, etc.
    # and the controller should perform the necessary conversions to wheel velocity or holo joint position.
    self.base_command_modes = [
        "planar_position",
        "planar_velocity",
        "wheel_velocity",
        "holo_joint_planar_position",
        "holo_joint_rel_planar_position",
    ]
    if self.robot_config.command_mode["base"] is not None:
        assert self.robot_config.command_mode["base"] in self.base_command_modes, (
            f"Base command mode {self.robot_config.command_mode['base']} not in {self.base_command_modes}"
        )
    self.base_command_mode = (
        "planar_position"
        if not self.robot_config.command_mode["base"]
        else self.robot_config.command_mode["base"]
    )
    if self.base_command_mode == "planar_position":
        base_controller = DiffDriveBasePoseController(
            self.robot_config,
            self.robot_view.get_move_group("base"),  # pyright: ignore[reportArgumentType]
        )
    elif self.base_command_mode == "holo_joint_rel_planar_position":
        base_controller = JointRelPosController(self.robot_view.get_move_group("base"))
    elif self.base_command_mode == "holo_joint_planar_position":
        base_controller = JointPosController(self.robot_view.get_move_group("base"))
    else:
        raise NotImplementedError(
            f"Base command mode {self.base_command_mode} not implemented yet."
        )

    # Head is fixed - no head actions are supported
    self.head_command_mode = self.robot_config.command_mode.get("head")
    assert self.head_command_mode is None, (
        "RBY1 head actuation is disabled. The head is fixed at init_qpos['head'] with optional "
        "randomization via init_qpos_noise_range['head']. "
        "Do not set command_mode['head'] to a non-None value."
    )

    # Torso command modes
    self.torso_command_modes = ["joint_position", "height"]
    self.torso_command_mode = (
        self.robot_config.command_mode.get("torso", "joint_position") or "joint_position"
    )
    assert self.torso_command_mode in self.torso_command_modes, (
        f"Torso command mode {self.torso_command_mode} not in {self.torso_command_modes}"
    )
    if self.torso_command_mode == "height":
        torso_controller = TorsoHeightJointPosController(
            self.robot_view.get_move_group("torso")
        )
    else:
        torso_controller = JointPosController(self.robot_view.get_move_group("torso"))

    self._controllers: dict[str, Controller] = {
        "base": base_controller,
        "torso": torso_controller,
        "left_arm": left_arm_controller,
        "right_arm": right_arm_controller,
        "left_gripper": left_gripper_controller,
        "right_gripper": right_gripper_controller,
    }
    assert set(self._controllers.keys()).issubset(set(self._robot_view.move_group_ids())), (
        "All controller keys must be move group IDs"
    )

arm_command_mode instance-attribute

arm_command_mode = 'joint_position' if not self.robot_config.command_mode['arm'] else self.robot_config.command_mode['arm']

arm_command_modes instance-attribute

arm_command_modes = ['joint_position', 'joint_rel_position', 'joint_velocity', 'ee_position', 'ee_velocity']

base_command_mode instance-attribute

base_command_mode = 'planar_position' if not self.robot_config.command_mode['base'] else self.robot_config.command_mode['base']

base_command_modes instance-attribute

base_command_modes = ['planar_position', 'planar_velocity', 'wheel_velocity', 'holo_joint_planar_position', 'holo_joint_rel_planar_position']

controllers property

controllers: dict[str, Controller]

One or more controllers for the robot joints / wheels / etc.

gripper_command_mode instance-attribute

gripper_command_mode = 'joint_position' if not self.robot_config.command_mode['gripper'] else self.robot_config.command_mode['gripper']

gripper_command_modes instance-attribute

gripper_command_modes = ['joint_position', 'joint_rel_position', 'joint_velocity']

head_command_mode instance-attribute

head_command_mode = self.robot_config.command_mode.get('head')

kinematics property

kinematics

kinematic solver for the robot

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

namespace property

namespace

robot namespace used to differentiate between one or multiple robots and the environment

parallel_kinematics property

parallel_kinematics

parallel kinematic solver for the robot

robot_config instance-attribute

robot_config = robot_config

robot_view property

robot_view

robot view for interfacing with joints / links / actuators

torso_command_mode instance-attribute

torso_command_mode = self.robot_config.command_mode.get('torso', 'joint_position') or 'joint_position'

torso_command_modes instance-attribute

torso_command_modes = ['joint_position', 'height']

add_robot_to_scene classmethod

add_robot_to_scene(robot_config: BaseRobotConfig, spec: MjSpec, prefix: str, pos: list[float], quat: list[float], randomize_textures: bool = False, strip_meshes: bool = False) -> None

Add a robot to a scene, taking care of any robot-specific considerations.

Parameters:

Name	Type	Description	Default
robot_config	BaseRobotConfig	The robot config, of the corresponding derived class (i.e. FrankaConfig for Franka, etc.)	required
spec	MjSpec	The scene to insert the robot into	required
prefix	str	The prefix to use for the robot, i.e. the namespace	required
pos	list[float]	The position to use for the robot, either 2d or 3d. If 2d, the z-coordinate is assumed to be 0.	required
quat	list[float]	The quaternion to use for the robot, in [w, x, y, z] format.	required
randomize_textures	bool	Whether to randomize the textures of the robot, if applicable. Not supported by all robots.	False
strip_meshes	bool	If True, remove all mesh geometries before insertion (for kinematics-only models).	False

Source code in molmo_spaces/robots/rby1.py

@classmethod
def add_robot_to_scene(
    cls,
    robot_config: BaseRobotConfig,
    spec: mj.MjSpec,
    prefix: str,
    pos: list[float],
    quat: list[float],
    randomize_textures: bool = False,
    strip_meshes: bool = False,
) -> None:
    assert prefix == "robot_0/", "RBY1 robot namespace must be 'robot_0/'"

    # RBY1's own MJCF wraps robot_model_root_name() ("base") inside an
    # intermediate body carrying a +0.005m Z offset (the same ground-clearance
    # constant used below for the "world" site). attach_body() only grafts the
    # specified root and its descendants, so that ancestor -- and its offset --
    # gets silently dropped, leaving the attached robot ~5mm too low. That's
    # enough to push its wheels past the named floor body into the raw "world"
    # ground plane, which check_robot_collision_in_current_pose doesn't
    # recognize as an exempt floor contact. Reapply the offset here.
    #
    # RBY1's MJCF also defines two mocap marker bodies (target_ee_pose_left/
    # right, used only to visualize task_config.viz_target_ee) as top-level
    # siblings of that same wrapper body, so they're dropped for the same
    # reason -- attach_body() only grafts "base"'s own subtree. Unlike the Z
    # offset, these aren't reattached: they share mesh assets (e.g. "EE_BODY")
    # with "base"'s own subtree, and MjSpec's attach_body raises on those
    # shared-asset IDs when attaching a second body from the same source spec.
    # See check_if_use_flip_side_handle/get_target_ee_pose's model.nmocap
    # guard for the corresponding graceful fallback.
    base_pos_3d = pos + [0.0] if len(pos) == 2 else list(pos)
    base_pos_3d[2] += 0.005

    super().add_robot_to_scene(
        robot_config=robot_config,
        spec=spec,
        prefix="",  # elements already have robot_0/ prefix in MJCF
        pos=base_pos_3d,
        quat=quat,
        randomize_textures=randomize_textures,
        strip_meshes=strip_meshes,
    )

    # RBY1 doesn't support insertion not at the origin or identity rotation
    # This can be fixed if it's worth the effort (see mobile franka for example)
    pos_check = pos + [0.0] if len(pos) == 2 else pos
    assert np.allclose(np.array(pos_check), [0, 0, 0]), "RBY1 insertion position is not zero!"
    assert np.allclose(np.array(quat), [1, 0, 0, 0]), "RBY1 insertion rotation is not identity!"

    def add_slider_act(
        name: str, ctrlrange: float, gainprm: float, biasprm: list[float], gear_idx: int
    ):
        act = spec.add_actuator(
            name=f"{prefix}{name}",
            target=f"{prefix}base_site",
            refsite=f"{prefix}world",
            ctrlrange=[-ctrlrange, ctrlrange],
            gear=[1 if i == gear_idx else 0 for i in range(6)],
            biastype=mj.mjtBias.mjBIAS_AFFINE,
            trntype=mj.mjtTrn.mjTRN_SITE,
        )
        act.gainprm[0] = gainprm
        act.biasprm[: len(biasprm)] = biasprm
        return act

    if getattr(robot_config, "use_holo_base"):  # noqa: B009
        spec.worldbody.add_site(name=f"{prefix}world", pos=[0, 0, 0.005], quat=[1, 0, 0, 0])
        add_slider_act("base_x_act", 25, 25000, [0, -25000, 0.5], 0)
        add_slider_act("base_y_act", 25, 25000, [0, -25000, 0.5], 1)
        add_slider_act("base_theta_act", np.pi, 5000, [0, -5000, 0.5], 5)

apply_action_noise

apply_action_noise(action: dict[str, Any]) -> dict[str, Any]

Apply action noise to the commanded action.

Extends the base class implementation to also apply base noise for RBY1's holonomic base commands.

Parameters:

Name	Type	Description	Default
action	dict[str, Any]	Action dict with move_group_id -> joint positions	required

Returns:

Type	Description
dict[str, Any]	Modified action dict with noise added

Source code in molmo_spaces/robots/rby1.py

def apply_action_noise(self, action: dict[str, Any]) -> dict[str, Any]:
    """Apply action noise to the commanded action.

    Extends the base class implementation to also apply base noise
    for RBY1's holonomic base commands.

    Args:
        action: Action dict with move_group_id -> joint positions

    Returns:
        Modified action dict with noise added
    """
    # TODO(wilbert): seems noise_config is not used at all for this part of rby1 code, we could
    # remove it instead of having to check, but for now will just assert and remove it in a
    # later commit
    noise_config = self.robot_config.action_noise_config
    if noise_config and not noise_config.enabled:
        return action

    # Apply arm noise via parent class
    noisy_action = super().apply_action_noise(action)

    # Apply base noise if base command is present and using holonomic planar mode
    if "base" in action and action["base"] is not None:
        if "holo_joint" in self.base_command_mode and "planar" in self.base_command_mode:
            commanded_base_pos = np.asarray(action["base"])
            noisy_base_pos = self._apply_base_noise(commanded_base_pos)
            noisy_action["base"] = noisy_base_pos

    return noisy_action

apply_control_overrides classmethod

apply_control_overrides(spec: MjSpec, robot_config: BaseRobotConfig)

Source code in molmo_spaces/robots/rby1.py

@classmethod
def apply_control_overrides(cls, spec: mj.MjSpec, robot_config: BaseRobotConfig):
    # the model root name already includes the hardcoded namespace
    tmp_robot_config = robot_config.model_copy(deep=True)
    tmp_robot_config.robot_namespace = ""
    super().apply_control_overrides(spec, tmp_robot_config)

compute_control

compute_control() -> None

Compute and set the control inputs to the robot actuators based on the current state and the targets set by the user. Must be called after update_control().

Source code in molmo_spaces/robots/abstract.py

def compute_control(self) -> None:
    """
    Compute and set the control inputs to the robot actuators based on the
    current state and the targets set by the user.
    Must be called after update_control().
    """
    for controller in self.controllers.values():
        ctrl_inputs = controller.compute_ctrl_inputs()
        controller.robot_move_group.ctrl = ctrl_inputs

create_robot_sensors

create_robot_sensors()

Get robot-specific sensors that should be registerd with a Task's sensor suite.

Source code in molmo_spaces/robots/rby1.py

def create_robot_sensors(self):
    return super().create_robot_sensors() + [
        RBY1GraspStateSensor(uuid="rby1_left_grasp_state", arm_side="left"),
        RBY1GraspStateSensor(uuid="rby1_right_grasp_state", arm_side="right"),
    ]

get_arm_move_group_ids

get_arm_move_group_ids() -> list[str]

RBY1 has two independent arms - each gets independent noise.

Source code in molmo_spaces/robots/rby1.py

def get_arm_move_group_ids(self) -> list[str]:
    """RBY1 has two independent arms - each gets independent noise."""
    return ["left_arm", "right_arm"]

get_joint_position

get_joint_position(move_group_ids: list[str]) -> ndarray

Get the current joint positions of the move groups

Source code in molmo_spaces/robots/rby1.py

def get_joint_position(self, move_group_ids: list[str]) -> np.ndarray:
    """Get the current joint positions of the move groups"""
    return np.concatenate(
        [
            self._robot_view.get_move_group(move_group_id).joint_pos.copy()
            for move_group_id in move_group_ids
        ]
    )

get_joint_ranges

get_joint_ranges(move_group_ids: list[str])

Get the joint ranges of the move groups

Source code in molmo_spaces/robots/rby1.py

def get_joint_ranges(self, move_group_ids: list[str]):
    """Get the joint ranges of the move groups"""
    joint_ranges = {}
    count = 0
    for move_group_id in move_group_ids:
        joint_ranges[move_group_id] = (
            count,
            count + self._robot_view.get_move_group(move_group_id).n_joints,
        )
        count += self._robot_view.get_move_group(move_group_id).n_joints
    return joint_ranges

get_world_pose_tf_mat

get_world_pose_tf_mat()

Get the robot's world pose transformation matrix.

Returns:

Type	Description
	np.ndarray: 4x4 transformation matrix for the robot base pose in world frame

Source code in molmo_spaces/robots/rby1.py

def get_world_pose_tf_mat(self):
    """Get the robot's world pose transformation matrix.

    Returns:
        np.ndarray: 4x4 transformation matrix for the robot base pose in world frame
    """
    return self.robot_view.get_move_group("base").pose  # pyright: ignore[reportAttributeAccessIssue]

last_unnoised_cmd_joint_pos

last_unnoised_cmd_joint_pos() -> dict[str, ndarray] | None

Get the last unnoised joint position commands for the robot.

Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action. Done signals are not included. Non-position controlled move groups are not included.

Returns:

Type	Description
dict[str, ndarray] | None	The last unnoised joint position commands for the robot, or None if no action has been set yet.

Source code in molmo_spaces/robots/abstract.py

def last_unnoised_cmd_joint_pos(self) -> dict[str, np.ndarray] | None:
    """Get the last unnoised joint position commands for the robot.

    Even if the commanded action was partial (due to e.g. noop), this will return the completely filled-in action.
    Done signals are not included.
    Non-position controlled move groups are not included.

    Returns:
        The last unnoised joint position commands for the robot, or None if no action has been set yet.
    """
    return deepcopy(self._last_unnoised_cmd_joint_pos)

reset

reset() -> None

Reset the robot to its initial state.

Source code in molmo_spaces/robots/rby1.py

def reset(self) -> None:
    """Reset the robot to its initial state."""

    init_qpos_dict = {key: np.array(val) for key, val in self.robot_config.init_qpos.items()}
    self.set_joint_pos(init_qpos_dict)

    # reset controllers
    for _, controller in self._controllers.items():
        controller.reset()

    # Set the head ctrl to match its qpos to prevent jerking at trajectory start.
    # The head has actuators but no controller, so we manually set ctrl = noop_ctrl.
    head_mg = self.robot_view.get_move_group("head")
    head_mg.ctrl = head_mg.noop_ctrl

robot_model_root_name staticmethod

robot_model_root_name() -> str

The name of the root body of the robot model. This is NOT necessarily the root body of the robot after insertion.

Source code in molmo_spaces/robots/rby1.py

@staticmethod
def robot_model_root_name() -> str:
    return "robot_0/base"

set_joint_pos

set_joint_pos(robot_joint_pos_dict: dict[str, ndarray]) -> None

Set all the robot's joint positions to the specified values.

Parameters:

Name	Type	Description	Default
robot_joint_pos_dict	dict[str, ndarray]	Dictionary containing joint positions for the robot based on the move groups ids.	required

Source code in molmo_spaces/robots/abstract.py

def set_joint_pos(self, robot_joint_pos_dict: dict[str, np.ndarray]) -> None:
    """Set all the robot's joint positions to the specified values.

    Args:
        robot_joint_pos_dict: Dictionary containing joint positions for the robot
            based on the move groups ids.
    """
    for mg_id, joint_pos in robot_joint_pos_dict.items():
        self.robot_view.get_move_group(mg_id).joint_pos = joint_pos

set_stationary

set_stationary() -> None

Set all controllers of the robot to stationary mode (if not already set), i.e., hold current positions or zero velocities.

Source code in molmo_spaces/robots/abstract.py

def set_stationary(self) -> None:
    """Set all controllers of the robot to stationary mode (if not already set),
    i.e., hold current positions or zero velocities."""
    for _, controller in self.controllers.items():
        if not controller.stationary:
            controller.set_to_stationary()

set_world_pose

set_world_pose(robot_world_pose: ndarray | list[float]) -> None

Set the robot's world pose to the specified pose in the world frame.

Parameters:

Name	Type	Description	Default
robot_world_pose	ndarray | list[float]	The pose of the robot in the world frame. If (4, 4) numpy array, it is assumed to be a transformation matrix. If (3,) list or numpy array, it is assumed to be (x, y, yaw). If (7,) list or numpy array, it is assumed to be (xyz + wxyz).	required

Source code in molmo_spaces/robots/abstract.py

def set_world_pose(self, robot_world_pose: np.ndarray | list[float]) -> None:
    """
    Set the robot's world pose to the specified pose in the world frame.

    Args:
        robot_world_pose: The pose of the robot in the world frame.
            If (4, 4) numpy array, it is assumed to be a transformation matrix.
            If (3,) list or numpy array, it is assumed to be (x, y, yaw).
            If (7,) list or numpy array, it is assumed to be (xyz + wxyz).
    """
    robot_world_pose = np.asarray(robot_world_pose)
    if robot_world_pose.shape == (4, 4):
        pose = robot_world_pose
    elif robot_world_pose.shape == (3,):
        pose = np.eye(4)
        pose[:2, 3] = robot_world_pose[:2]
        pose[:3, :3] = R.from_euler("Z", robot_world_pose[2]).as_matrix()
    elif robot_world_pose.shape == (7,):
        pose = np.eye(4)
        pose[:3, 3] = robot_world_pose[:3]
        pose[:3, :3] = R.from_quat(robot_world_pose[3:], scalar_first=True).as_matrix()
    else:
        raise ValueError(f"Invalid robot world pose shape: {robot_world_pose.shape}")

    self.robot_view.base.pose = pose

update_control

update_control(action_command_dict: dict[str, ndarray]) -> None

Update the control targets to the robot based on the provided action commands. Does not set the control inputs to the robot actuators. See compute_control().

Parameters:

Name	Type	Description	Default
action_command_dict	dict[str, ndarray]	Dictionary containing action commands for the robot based on the move groups ids to be used.	required

Source code in molmo_spaces/robots/abstract.py

def update_control(self, action_command_dict: dict[str, np.ndarray]) -> None:
    """
    Update the control targets to the robot based on the provided action commands.
    Does not set the control inputs to the robot actuators. See compute_control().

    Args:
        action_command_dict: Dictionary containing action commands for the robot
                             based on the move groups ids to be used.
    """
    action_command_dict = self._apply_action_noise_and_save_unnoised_cmd_jp(action_command_dict)

    for mg_id, controller in self.controllers.items():
        if mg_id in action_command_dict and action_command_dict[mg_id] is not None:
            controller.set_target(action_command_dict[mg_id])
        elif not controller.stationary:
            controller.set_to_stationary()

robot_views

Modules:

Name	Description
abstract	This module defines the core abstractions for representing and controlling robots in MuJoCo.
bimanual_yam_view	Implementation of the bimanual YAM robot model.
franka_cap_view
franka_droid_view
franka_fr3_view	Implementation of the Franka FR3 robot model.
g1_view	Implementation of the Unitree G1 robot model.
i2rt_yam_view	Implementation of the i2rt YAM robot model.
mobile_franka_droid_view
rby1_view	Implementation of the RBY1 robot model.
rum_gripper_view

abstract

This module defines the core abstractions for representing and controlling robots in MuJoCo. The architecture is based on a hierarchical structure where a RobotView contains multiple MoveGroups, each representing an atomic collection of joints and actuators.

Classes:

Name	Description
FreeJointRobotBaseGroup	A RobotBase that uses a free joint to represent its pose.
GripperGroup
HoloJointsRobotBaseGroup	A RobotBase that uses virtual holonomic joints to represent its pose.
ImmobileRobotBaseGroup	A RobotBase that is immobile and does not have a mocap body.
MJCFFrameMixin	Mixin for move groups that represent the leaf frame as a body or site in the MJCF model.
MocapRobotBaseGroup	A RobotBase that uses a mocap body to represent its pose.
MoveGroup	Base class for any collection of joints and actuators in a robot.
RobotBaseGroup	A MoveGroup that controls the overall pose of the robot.
RobotView	Top-level class that contains and manages multiple MoveGroups.
SimplyActuatedMoveGroup	A SimplyActuatedMoveGroup is a move group with a 1:1 mapping between joints, actuators, and position/velocity addresses.

Attributes:

Name	Type	Description
RobotViewFactory	TypeAlias	Factory function that creates a RobotView from a MjData and a robot namespace

RobotViewFactory module-attribute

RobotViewFactory: TypeAlias = Callable[[mj.MjData, str], RobotView]

Factory function that creates a RobotView from a MjData and a robot namespace

FreeJointRobotBaseGroup

FreeJointRobotBaseGroup(mj_data: MjData, base_joint_id: int, joint_ids: list[int], actuator_ids: list[int], floating=False)

Bases: RobotBaseGroup

A RobotBase that uses a free joint to represent its pose.

This implementation uses MuJoCo's free joint type to represent the base pose, which allows for full 6-DOF control of the robot's position and orientation.

Initialize a FreeJointRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
base_joint_id	int	The ID of the free joint that represents the base pose	required
joint_ids	list[int]	List of additional joint IDs that belong to this base	required
actuator_ids	list[int]	List of actuator IDs that control the base	required
floating		Whether the base is floating (concretely, whether IK should not constrain it to be on the ground)	False

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
floating		Whether the base is floating.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: mj.MjData,
    base_joint_id: int,
    joint_ids: list[int],
    actuator_ids: list[int],
    floating=False,
):
    """Initialize a FreeJointRobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        base_joint_id: The ID of the free joint that represents the base pose
        joint_ids: List of additional joint IDs that belong to this base
        actuator_ids: List of actuator IDs that control the base
        floating: Whether the base is floating (concretely, whether IK should not constrain it to be on the ground)
    """
    base_body_id = mj_data.model.jnt_bodyid[base_joint_id]
    super().__init__(mj_data, [base_joint_id] + joint_ids, actuator_ids, base_body_id)
    assert self.mj_model.jnt_type[base_joint_id] == mj.mjtJoint.mjJNT_FREE
    self._base_joint_id = base_joint_id
    self._floating = floating

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

floating property

floating

Whether the base is floating.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl abstractmethod property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

GripperGroup

GripperGroup(mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int, robot_base_group: RobotBaseGroup | None = None)

Bases: MoveGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: mj.MjData,
    joint_ids: list[int],
    actuator_ids: list[int],
    root_body_id: int,
    robot_base_group: RobotBaseGroup | None = None,
) -> None:
    """Initialize a MoveGroup.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        joint_ids: List of joint IDs that belong to this move group
        actuator_ids: List of actuator IDs that control the joints
        root_body_id: The ID of the root body of this move group
        robot_base_group: The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data

    self._joint_ids = joint_ids
    self._robot_base_group = robot_base_group
    self._root_body_id = root_body_id
    self._name: str | None = None

    self._joint_posadr: list[int] = []
    self._joint_veladr: list[int] = []
    for i in joint_ids:
        n_pos_dim = 1
        n_vel_dim = 1
        if self.mj_model.jnt_type[i] == mj.mjtJoint.mjJNT_FREE:
            n_pos_dim = 7
            n_vel_dim = 6
        elif self.mj_model.jnt_type[i] == mj.mjtJoint.mjJNT_BALL:
            n_pos_dim = 4
            n_vel_dim = 3
        self._joint_posadr.extend(
            list(range(self.mj_model.jnt_qposadr[i], self.mj_model.jnt_qposadr[i] + n_pos_dim))
        )
        self._joint_veladr.extend(
            list(range(self.mj_model.jnt_dofadr[i], self.mj_model.jnt_dofadr[i] + n_vel_dim))
        )

    self._actuator_ids = actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist abstractmethod property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range abstractmethod property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world abstractmethod property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world abstractmethod property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian abstractmethod

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def get_jacobian(self) -> np.ndarray:
    """Returns the (6, model.nv) jacobian of the move group.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    raise NotImplementedError

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open abstractmethod

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def set_gripper_ctrl_open(self, open: bool) -> None:
    """Set the gripper commanded position to be fully open or closed.

    Args:
        open: True to open the gripper, False to close it
    """
    raise NotImplementedError

HoloJointsRobotBaseGroup

HoloJointsRobotBaseGroup(mj_data: MjData, world_site_id: int, holo_base_site_id: int, joint_ids: list[int], actuator_ids: list[int], root_body_id: int)

Bases: RobotBaseGroup, SimplyActuatedMoveGroup

A RobotBase that uses virtual holonomic joints to represent its pose.

Assumes three virtual holonomic joints for x, y, and theta control.

Initialize a HoloJointsRobotBase that has virtual holonomic joints and uses site control.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
world_site_id	int	The ID of the world site	required
holo_base_site_id	int	The ID of the site that represents the holonomic base pose	required
joint_ids	list[int]	List of joint IDs that belong to the virtual holonomic base pose. NOTE: Assumed order is [x, y, theta].	required
actuator_ids	list[int]	List of actuator IDs that control the base	required
root_body_id	int	The ID of the body that represents the robot base	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the holonomic base actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: mj.MjData,
    world_site_id: int,
    holo_base_site_id: int,
    joint_ids: list[int],
    actuator_ids: list[int],
    root_body_id: int,
):
    """Initialize a HoloJointsRobotBase that has virtual holonomic joints and uses site control.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        world_site_id: The ID of the world site
        holo_base_site_id: The ID of the site that represents the holonomic base pose
        joint_ids: List of joint IDs that belong to the virtual holonomic base pose.
                   NOTE: Assumed order is [x, y, theta].
        actuator_ids: List of actuator IDs that control the base
        root_body_id: The ID of the body that represents the robot base
    """
    super().__init__(mj_data, joint_ids, actuator_ids, root_body_id)

    assert len(joint_ids) == 3, "HoloJointsRobotBaseGroup must have 3 joints (x, y, theta)"
    assert all(
        self.mj_model.jnt_type[jid] == mj.mjtJoint.mjJNT_SLIDE for jid in joint_ids[:2]
    ), "x, y joints must be slide joints"
    assert self.mj_model.jnt_type[joint_ids[2]] == mj.mjtJoint.mjJNT_HINGE, (
        "theta joint must be hinge"
    )
    assert len(actuator_ids) == 3, (
        "HoloJointsRobotBaseGroup must have 3 actuators (x, y, theta)"
    )

    self._world_site_id = world_site_id
    self._holo_base_site_id = holo_base_site_id

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the holonomic base actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._holo_base_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

ImmobileRobotBaseGroup

ImmobileRobotBaseGroup(mj_data: MjData, robot_base_body_id: int)

Bases: RobotBaseGroup

A RobotBase that is immobile and does not have a mocap body.

This generally shouldn't be used, and MocapRobotBaseGroup should be preferred. If a scene is badly constructed and the robot base is not a mocap body, this might be necessary.

Initialize a ImmobileRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the simulation state	required
robot_base_body_id	int	The ID of the body that represents the robot base	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(self, mj_data: mj.MjData, robot_base_body_id: int) -> None:
    """Initialize a ImmobileRobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the simulation state
        robot_base_body_id: The ID of the body that represents the robot base
    """
    super().__init__(mj_data, [], [], robot_base_body_id)
    self._robot_base_body_id = robot_base_body_id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

MJCFFrameMixin

Bases: ABC

Mixin for move groups that represent the leaf frame as a body or site in the MJCF model.

Since this mixin provides a get_jacobian() implementation, inheriting classes must

put this mixin first in the inheritance chain to satisfy the MRO.

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.

Attributes:

Name	Type	Description
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_world	ndarray
leaf_frame_type	Literal['site', 'body']	The type of the leaf frame.

leaf_frame_id abstractmethod property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type abstractmethod property

leaf_frame_type: Literal['site', 'body']

The type of the leaf frame.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

MocapRobotBaseGroup

MocapRobotBaseGroup(mj_data: MjData, robot_base_body_id: int)

Bases: RobotBaseGroup

A RobotBase that uses a mocap body to represent its pose.

Initialize a MocapRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the simulation state	required
robot_base_body_id	int	The ID of the mocap body that represents the robot base. Not the Mocap ID!	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(self, mj_data: mj.MjData, robot_base_body_id: int):
    """Initialize a MocapRobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the simulation state
        robot_base_body_id: The ID of the mocap body that represents the robot base. Not the Mocap ID!
    """
    robot_base_mocap_id = mj_data.model.body_mocapid[robot_base_body_id]
    super().__init__(mj_data, [], [], robot_base_body_id)
    self._robot_base_mocap_id = robot_base_mocap_id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

MoveGroup

MoveGroup(mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int, robot_base_group: RobotBaseGroup | None = None)

Bases: ABC

Base class for any collection of joints and actuators in a robot.

A MoveGroup represents an atomic collection of robotic joints and their associated actuators. It maintains a kinematic tree between a root frame and a leaf frame, and provides methods to control and query the state of its joints and actuators.

The class handles the low-level details of interfacing with MuJoCo's state vectors (qpos, qvel) and control signals (ctrl), providing a clean interface for higher-level control.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: mj.MjData,
    joint_ids: list[int],
    actuator_ids: list[int],
    root_body_id: int,
    robot_base_group: RobotBaseGroup | None = None,
) -> None:
    """Initialize a MoveGroup.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        joint_ids: List of joint IDs that belong to this move group
        actuator_ids: List of actuator IDs that control the joints
        root_body_id: The ID of the root body of this move group
        robot_base_group: The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data

    self._joint_ids = joint_ids
    self._robot_base_group = robot_base_group
    self._root_body_id = root_body_id
    self._name: str | None = None

    self._joint_posadr: list[int] = []
    self._joint_veladr: list[int] = []
    for i in joint_ids:
        n_pos_dim = 1
        n_vel_dim = 1
        if self.mj_model.jnt_type[i] == mj.mjtJoint.mjJNT_FREE:
            n_pos_dim = 7
            n_vel_dim = 6
        elif self.mj_model.jnt_type[i] == mj.mjtJoint.mjJNT_BALL:
            n_pos_dim = 4
            n_vel_dim = 3
        self._joint_posadr.extend(
            list(range(self.mj_model.jnt_qposadr[i], self.mj_model.jnt_qposadr[i] + n_pos_dim))
        )
        self._joint_veladr.extend(
            list(range(self.mj_model.jnt_dofadr[i], self.mj_model.jnt_dofadr[i] + n_vel_dim))
        )

    self._actuator_ids = actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world abstractmethod property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl abstractmethod property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world abstractmethod property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian abstractmethod

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def get_jacobian(self) -> np.ndarray:
    """Returns the (6, model.nv) jacobian of the move group.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    raise NotImplementedError

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RobotBaseGroup

RobotBaseGroup(mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int)

Bases: MoveGroup

A MoveGroup that controls the overall pose of the robot.

A RobotBase represents the base of a robot, which can be either mobile (e.g., wheeled) or immobile (e.g., fixed to a table). It provides methods to control and query the overall pose of the robot in the world frame.

Initialize a RobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this base	required
actuator_ids	list[int]	List of actuator IDs that control the base	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self, mj_data: mj.MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int
):
    """Initialize a RobotBase.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        joint_ids: List of joint IDs that belong to this base
        actuator_ids: List of actuator IDs that control the base
    """
    super().__init__(mj_data, joint_ids, actuator_ids, root_body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl abstractmethod property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose abstractmethod property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian abstractmethod

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def get_jacobian(self) -> np.ndarray:
    """Returns the (6, model.nv) jacobian of the move group.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    raise NotImplementedError

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RobotView

RobotView(mj_data: MjData, move_groups: dict[str, MoveGroup])

Bases: ABC

Top-level class that contains and manages multiple MoveGroups.

A RobotView represents a complete robot, composed of multiple MoveGroups (arms, base, etc.). It provides a unified interface to control and query the state of all move groups, and handles the coordination between different parts of the robot.

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	RobotBaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(self, mj_data: mj.MjData, move_groups: dict[str, MoveGroup]) -> None:
    """Initialize a RobotView.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        move_groups: Dictionary mapping move group names to their implementations
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data
    self._move_groups = move_groups
    for mg_name, mg in move_groups.items():
        assert mg._name is None, f"Move group {mg_name} already has a name {mg._name}"
        mg._name = mg_name

base abstractmethod property

base: RobotBaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name abstractmethod property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

SimplyActuatedMoveGroup

SimplyActuatedMoveGroup(mj_data: MjData, joint_ids: list[int], actuator_ids: list[int], root_body_id: int, robot_base_group: RobotBaseGroup | None = None)

Bases: MoveGroup

A SimplyActuatedMoveGroup is a move group with a 1:1 mapping between joints, actuators, and position/velocity addresses.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def __init__(
    self,
    mj_data: mj.MjData,
    joint_ids: list[int],
    actuator_ids: list[int],
    root_body_id: int,
    robot_base_group: RobotBaseGroup | None = None,
) -> None:
    """Initialize a MoveGroup.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        joint_ids: List of joint IDs that belong to this move group
        actuator_ids: List of actuator IDs that control the joints
        root_body_id: The ID of the root body of this move group
        robot_base_group: The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.
    """
    self.mj_model = mj_data.model
    self.mj_data = mj_data

    self._joint_ids = joint_ids
    self._robot_base_group = robot_base_group
    self._root_body_id = root_body_id
    self._name: str | None = None

    self._joint_posadr: list[int] = []
    self._joint_veladr: list[int] = []
    for i in joint_ids:
        n_pos_dim = 1
        n_vel_dim = 1
        if self.mj_model.jnt_type[i] == mj.mjtJoint.mjJNT_FREE:
            n_pos_dim = 7
            n_vel_dim = 6
        elif self.mj_model.jnt_type[i] == mj.mjtJoint.mjJNT_BALL:
            n_pos_dim = 4
            n_vel_dim = 3
        self._joint_posadr.extend(
            list(range(self.mj_model.jnt_qposadr[i], self.mj_model.jnt_qposadr[i] + n_pos_dim))
        )
        self._joint_veladr.extend(
            list(range(self.mj_model.jnt_dofadr[i], self.mj_model.jnt_dofadr[i] + n_vel_dim))
        )

    self._actuator_ids = actuator_ids

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world abstractmethod property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world abstractmethod property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian abstractmethod

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

@abstractmethod
def get_jacobian(self) -> np.ndarray:
    """Returns the (6, model.nv) jacobian of the move group.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    raise NotImplementedError

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

bimanual_yam_view

Implementation of the bimanual YAM robot model.

The bimanual YAM consists of two 6-DOF YAM arms with parallel grippers, positioned side by side (44cm apart), both facing forward.

Each arm component is implemented as a MoveGroup (left_arm, right_arm, left_gripper, right_gripper), with the overall robot structure managed by the BimanualYamRobotView class.

Classes:

Name	Description
BimanualYamArmGroup	6-DOF arm group for one side of the bimanual YAM robot.
BimanualYamBaseGroup	Base group for the bimanual YAM robot using mocap body control.
BimanualYamGripperGroup	Parallel gripper group for one side of the bimanual YAM robot.
BimanualYamRobotView	Robot view for the bimanual YAM (two 6-DOF arms with parallel grippers).

BimanualYamArmGroup

BimanualYamArmGroup(mj_data: MjData, side: Literal['left', 'right'], base_group: BimanualYamBaseGroup, namespace: str = '', grasp_site_name: str = 'grasp_site')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

6-DOF arm group for one side of the bimanual YAM robot.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
side	str
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base_group: BimanualYamBaseGroup,
    namespace: str = "",
    grasp_site_name: str = "grasp_site",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    self._side = side
    # Prefix for this arm: namespace + side_
    self._arm_prefix = f"{namespace}{side}_"

    # 6 arm joints: joint1 through joint6
    joint_ids = [model.joint(f"{self._arm_prefix}joint{i + 1}").id for i in range(6)]
    # 6 arm actuators with the same names
    act_ids = [model.actuator(f"{self._arm_prefix}joint{i + 1}").id for i in range(6)]
    self._arm_root_id = model.body(f"{self._arm_prefix}arm").id
    self._ee_site_id = model.site(f"{self._arm_prefix}{grasp_site_name}").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base_group)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

side property

side: str

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

BimanualYamBaseGroup

BimanualYamBaseGroup(mj_data: MjData, namespace: str = '')

Bases: MocapRobotBaseGroup

Base group for the bimanual YAM robot using mocap body control.

The mocap body is created by add_robot_to_scene and serves as the root for both arms.

Initialize a MocapRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the simulation state	required
robot_base_body_id	int	The ID of the mocap body that represents the robot base. Not the Mocap ID!	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    body_id: int = mj_data.model.body(f"{namespace}base").id
    super().__init__(mj_data, body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

BimanualYamGripperGroup

BimanualYamGripperGroup(mj_data: MjData, side: Literal['left', 'right'], base_group: BimanualYamBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

Parallel gripper group for one side of the bimanual YAM robot.

The YAM gripper has two coupled fingers (left_finger and right_finger) controlled by a single actuator. The fingers move in opposite directions due to an equality constraint.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set gripper to fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	Distance between fingers.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
side	str
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base_group: BimanualYamBaseGroup,
    namespace: str = "",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    self._side = side
    # Prefix for this gripper: namespace + side_
    self._gripper_prefix = f"{namespace}{side}_"

    # Two coupled finger joints
    joint_ids = [
        model.joint(f"{self._gripper_prefix}left_finger").id,
        model.joint(f"{self._gripper_prefix}right_finger").id,
    ]
    # Single gripper actuator controls left_finger (right follows via equality constraint)
    act_ids = [model.actuator(f"{self._gripper_prefix}gripper").id]
    root_body_id = model.body(f"{self._gripper_prefix}link_6").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{self._gripper_prefix}grasp_site").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

Distance between fingers.

Since fingers are coupled in opposite directions (right = -left), the total opening is the sum of absolute positions.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers.

From XML: - left_finger range: -0.00205 to 0.037524 - right_finger range: -0.037524 to 0.00205 (mirrored) - Closed (both at 0): dist = 0 - Open (left at 0.037524, right at -0.037524): dist ~= 0.075

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

side property

side: str

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set gripper to fully open or closed.

From XML: gripper actuator ctrlrange is 0.0 to 0.041 - 0.0 = closed - 0.041 = open

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    """Set gripper to fully open or closed.

    From XML: gripper actuator ctrlrange is 0.0 to 0.041
    - 0.0 = closed
    - 0.041 = open
    """
    self.ctrl = [0.041 if open else 0.0]

BimanualYamRobotView

BimanualYamRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Robot view for the bimanual YAM (two 6-DOF arms with parallel grippers).

Move groups: - base: Mocap body controlling both arms - left_arm: Left 6-DOF arm - right_arm: Right 6-DOF arm - left_gripper: Left parallel gripper - right_gripper: Right parallel gripper

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	BimanualYamBaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/bimanual_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = BimanualYamBaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "left_arm": BimanualYamArmGroup(mj_data, "left", base, namespace=namespace),
        "right_arm": BimanualYamArmGroup(mj_data, "right", base, namespace=namespace),
        "left_gripper": BimanualYamGripperGroup(mj_data, "left", base, namespace=namespace),
        "right_gripper": BimanualYamGripperGroup(mj_data, "right", base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: BimanualYamBaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

franka_cap_view

Classes:

Name	Description
CAPGripperGroup
FrankaCAPRobotView

CAPGripperGroup

CAPGripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_cap_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}gripper/finger_left_joint").id,
        model.joint(f"{namespace}gripper/finger_right_joint").id,
    ]
    act_ids = [model.actuator(f"{namespace}gripper/fingers_actuator").id]
    root_body_id = model.body(f"{namespace}gripper/base").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}gripper/grasp_site").id
    self._finger_1_geom_id = model.geom(f"{namespace}gripper/left_pad2").id
    self._finger_2_geom_id = model.geom(f"{namespace}gripper/right_pad2").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/franka_cap_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [0 if open else 255]

FrankaCAPRobotView

FrankaCAPRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FrankaFR3BaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/franka_cap_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FrankaFR3BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": FrankaFR3ArmGroup(
            mj_data, base, namespace=namespace, grasp_site_name="gripper/grasp_site"
        ),
        "gripper": CAPGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FrankaFR3BaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

franka_droid_view

Classes:

Name	Description
FloatingRobotiq2f85BaseGroup
FloatingRobotiq2f85RobotView
FloatingRobotiqGripperGroup
FrankaDroidRobotView
RobotIQGripperGroup

FloatingRobotiq2f85BaseGroup

FloatingRobotiq2f85BaseGroup(mj_data: MjData, namespace: str = '')

Bases: FreeJointRobotBaseGroup

Initialize a FreeJointRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
base_joint_id	int	The ID of the free joint that represents the base pose	required
joint_ids	list[int]	List of additional joint IDs that belong to this base	required
actuator_ids	list[int]	List of actuator IDs that control the base	required
floating		Whether the base is floating (concretely, whether IK should not constrain it to be on the ground)	False

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
is_mobile	Whether this base is mobile (has actuators).
n_actuators	Number of actuators in this move group.

Attributes:

Name	Type	Description
ctrl		Current control signals for the actuators.
ctrl_limits		Control limits (min, max) for each actuator.
floating		Whether the base is floating.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    model = mj_data.model
    base_joint_id = model.joint(f"{namespace}robotiq_2f85_free_joint").id
    self._target_pose_body = create_mlspaces_body(mj_data, f"{namespace}target_ee_pose")
    super().__init__(mj_data, base_joint_id, [], [], floating=True)

ctrl property writable

ctrl

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits

Control limits (min, max) for each actuator.

floating property

floating

Whether the base is floating.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

is_mobile

is_mobile()

Whether this base is mobile (has actuators).

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def is_mobile(self):
    return True

n_actuators

n_actuators()

Number of actuators in this move group.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def n_actuators(self):
    return 7

FloatingRobotiq2f85RobotView

FloatingRobotiq2f85RobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FloatingRobotiq2f85BaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FloatingRobotiq2f85BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "gripper": FloatingRobotiqGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FloatingRobotiq2f85BaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

FloatingRobotiqGripperGroup

FloatingRobotiqGripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: RobotIQGripperGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}left_driver_joint").id,
        model.joint(f"{namespace}right_driver_joint").id,
    ]
    act_ids = [model.actuator(f"{namespace}fingers_actuator").id]
    root_body_id = model.body(f"{namespace}base").id
    GripperGroup.__init__(self, mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id
    self._finger_1_geom_id = model.geom(f"{namespace}left_pad2").id
    self._finger_2_geom_id = model.geom(f"{namespace}right_pad2").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [0 if open else 255]

FrankaDroidRobotView

FrankaDroidRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FrankaFR3BaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FrankaFR3BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": FrankaFR3ArmGroup(
            mj_data, base, namespace=namespace, grasp_site_name="gripper/grasp_site"
        ),
        "gripper": RobotIQGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FrankaFR3BaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

RobotIQGripperGroup

RobotIQGripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}gripper/left_driver_joint").id,
        model.joint(f"{namespace}gripper/right_driver_joint").id,
    ]
    act_ids = [model.actuator(f"{namespace}gripper/fingers_actuator").id]
    root_body_id = model.body(f"{namespace}gripper/base").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}gripper/grasp_site").id
    self._finger_1_geom_id = model.geom(f"{namespace}gripper/left_pad2").id
    self._finger_2_geom_id = model.geom(f"{namespace}gripper/right_pad2").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/franka_droid_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [0 if open else 255]

franka_fr3_view

Implementation of the Franka FR3 robot model.

The Franka FR3 is a single-arm 7-DOF robot with a gripper.

Each component is implemented as a MoveGroup, with the overall robot structure managed by the FrankaFR3RobotView class.

Classes:

Name	Description
FrankaFR3ArmGroup
FrankaFR3BaseGroup
FrankaFR3GripperGroup
FrankaFR3RobotView

FrankaFR3ArmGroup

FrankaFR3ArmGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '', grasp_site_name: str = 'grasp_site')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(
    self,
    mj_data: MjData,
    base_group: FrankaFR3BaseGroup,
    namespace: str = "",
    grasp_site_name: str = "grasp_site",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [model.joint(f"{namespace}fr3_joint{i + 1}").id for i in range(7)]
    act_ids = [model.actuator(f"{namespace}fr3_joint{i + 1}").id for i in range(7)]
    self._arm_root_id = model.body(f"{namespace}fr3_link0").id
    self._ee_site_id = model.site(f"{namespace}{grasp_site_name}").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base_group)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

FrankaFR3BaseGroup

FrankaFR3BaseGroup(mj_data: MjData, namespace: str = '')

Bases: MocapRobotBaseGroup

Initialize a MocapRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the simulation state	required
robot_base_body_id	int	The ID of the mocap body that represents the robot base. Not the Mocap ID!	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    body_id: int = mj_data.model.body(f"{namespace}base").id
    super().__init__(mj_data, body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

FrankaFR3GripperGroup

FrankaFR3GripperGroup(mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(
    self, mj_data: MjData, base_group: FrankaFR3BaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}finger_joint1").id,
        model.joint(f"{namespace}finger_joint2").id,
    ]
    act_ids = [model.actuator(f"{namespace}panda_hand").id]
    root_body_id = model.body(f"{namespace}hand").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = [255 if open else 0]

FrankaFR3RobotView

FrankaFR3RobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FrankaFR3BaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/franka_fr3_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = FrankaFR3BaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": FrankaFR3ArmGroup(mj_data, base, namespace=namespace),
        "gripper": FrankaFR3GripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FrankaFR3BaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

g1_view

Implementation of the Unitree G1 robot model.

The G1 (as configured here) is a bipedal humanoid with: - A free-floating pelvis (real 6-DOF dynamics, not a kinematic/wheeled base) - 12 leg joints + a 3-DOF waist/torso, combined into one 15-DOF move group driven by a whole-body walking controller (see LegsWaistController) - Two 7-DOF arms (left arm has no gripper and is commanded to its natural hanging pose; right arm has a dexterous gripper) - A single right-hand gripper, tendon-actuated, with two mechanically coupled finger joints (joint2 = joint1, enforced by an MJCF constraint)

Each component is implemented as a MoveGroup, with the overall robot structure managed by the G1RobotView class. There is no independent head MoveGroup (the head/cameras are rigidly mounted to the torso) and no left-hand gripper (no hardware exists for it).

Classes:

Name	Description
G1ArmGroup	One of the G1's 7-DOF arms, excluding the gripper.
G1BaseGroup	The G1's free-floating pelvis. No wheels, no holonomic joints -- true
G1GripperGroup	The G1's right-hand dexterous gripper.
G1LegsWaistGroup	The 12 leg joints + 3-DOF waist as one 15-DOF group: the whole-body
G1RobotView	Implementation of the complete G1 robot (whole-body walking, see LegsWaistController).

Attributes:

Name	Type	Description
ARM_JOINT_SUFFIXES
GRIPPER_CLOSED
GRIPPER_OPEN
LEGS_WAIST_JOINT_SUFFIXES

ARM_JOINT_SUFFIXES module-attribute

ARM_JOINT_SUFFIXES = _ARM_JOINT_SUFFIXES

GRIPPER_CLOSED module-attribute

GRIPPER_CLOSED = 0.0245

GRIPPER_OPEN module-attribute

GRIPPER_OPEN = -0.0222

LEGS_WAIST_JOINT_SUFFIXES module-attribute

LEGS_WAIST_JOINT_SUFFIXES = _LEG_JOINT_SUFFIXES + _WAIST_JOINT_SUFFIXES

G1ArmGroup

G1ArmGroup(mj_data: MjData, side: str, base: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

One of the G1's 7-DOF arms, excluding the gripper.

The left arm has no gripper hardware and is normally commanded to its natural hanging pose; the right arm carries the dexterous gripper.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
side
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def __init__(
    self, mj_data: MjData, side: str, base: RobotBaseGroup, namespace: str = ""
) -> None:
    model = mj_data.model
    self.side = side
    joint_ids = [model.joint(f"{namespace}{side}_{n}").id for n in _ARM_JOINT_SUFFIXES]
    act_ids = [model.actuator(f"{namespace}walk_{side}_{n}").id for n in _ARM_JOINT_SUFFIXES]
    self._ee_site_id = model.site(f"{namespace}{side}_grasp").id
    self._arm_root_id = model.body(f"{namespace}{side}_shoulder_pitch_link").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

side instance-attribute

side = side

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

G1BaseGroup

G1BaseGroup(mj_data: MjData, namespace: str = '')

Bases: FreeJointRobotBaseGroup

The G1's free-floating pelvis. No wheels, no holonomic joints -- true 6-DOF rigid-body dynamics, integrated directly by MuJoCo's physics.

Initialize a FreeJointRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
base_joint_id	int	The ID of the free joint that represents the base pose	required
joint_ids	list[int]	List of additional joint IDs that belong to this base	required
actuator_ids	list[int]	List of actuator IDs that control the base	required
floating		Whether the base is floating (concretely, whether IK should not constrain it to be on the ground)	False

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
floating		Whether the base is floating.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    model = mj_data.model
    base_joint_id = model.joint(f"{namespace}floating_base_joint").id
    super().__init__(mj_data, base_joint_id, [], [])

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

floating property

floating

Whether the base is floating.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

G1GripperGroup

G1GripperGroup(mj_data: MjData, base: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

The G1's right-hand dexterous gripper.

Two finger joints (right_Joint1_1, right_Joint2_1) are mechanically coupled by an MJCF constraint enforcing joint2 = joint1 (unlike RBY1's gripper, which couples finger2 = -finger1). One tendon-driven position actuator (right_grip) drives both.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def __init__(self, mj_data: MjData, base: RobotBaseGroup, namespace: str = "") -> None:
    model = mj_data.model
    joint_ids = [
        model.joint(f"{namespace}right_Joint1_1").id,
        model.joint(f"{namespace}right_Joint2_1").id,
    ]
    act_ids = [model.actuator(f"{namespace}right_grip").id]
    self._ee_site_id = model.site(f"{namespace}right_grasp").id
    root_body_id = model.body(f"{namespace}right_dex_base").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/g1_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = np.array([GRIPPER_OPEN if open else GRIPPER_CLOSED])

G1LegsWaistGroup

G1LegsWaistGroup(mj_data: MjData, base: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

The 12 leg joints + 3-DOF waist as one 15-DOF group: the whole-body controller computes one coupled law across both. Joint order must match the walking policy's training-time order (legs first, then waist).

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def __init__(self, mj_data: MjData, base: RobotBaseGroup, namespace: str = "") -> None:
    model = mj_data.model
    suffixes = _LEG_JOINT_SUFFIXES + _WAIST_JOINT_SUFFIXES
    joint_ids = [model.joint(f"{namespace}{n}").id for n in suffixes]
    act_ids = [model.actuator(f"{namespace}walk_{n}").id for n in suffixes]
    self._pelvis_id = model.body(f"{namespace}pelvis").id
    self._waist_leaf_id = model.body(f"{namespace}torso_link").id
    super().__init__(mj_data, joint_ids, act_ids, self._pelvis_id, base)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

G1RobotView

G1RobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Implementation of the complete G1 robot (whole-body walking, see LegsWaistController).

No head move group (head/cameras are rigidly mounted to the torso) and no left_gripper move group (no hardware exists for it).

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
distance_to	Calculate the planar (x, y, theta) distance from the base's current pose to a target pose.
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Exclude legs/waist, but not the pelvis, from generic arm-reach IK.
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
is_close_to	Is the planar base pose within threshold of the target. The default
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base		The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = G1BaseGroup(mj_data, namespace=namespace)
    move_groups: dict[str, MoveGroup] = {
        "base": base,
        "legs_waist": G1LegsWaistGroup(mj_data, base, namespace=namespace),
        "left_arm": G1ArmGroup(mj_data, "left", base, namespace=namespace),
        "right_arm": G1ArmGroup(mj_data, "right", base, namespace=namespace),
        "right_gripper": G1GripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

distance_to

distance_to(move_group_ids: list[str], target_pose: list) -> float

Calculate the planar (x, y, theta) distance from the base's current pose to a target pose.

The pelvis's pose is a full 6-DOF transform (see FreeJointRobotBaseGroup.pose), unlike a holonomic base's native [x, y, theta] joints, so we project it down to the world-frame yaw for comparison against the [x, y, theta] waypoints the A* nav planner uses.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def distance_to(self, move_group_ids: list[str], target_pose: list) -> float:
    """Calculate the planar (x, y, theta) distance from the base's current pose to a target pose.

    The pelvis's pose is a full 6-DOF transform (see FreeJointRobotBaseGroup.pose),
    unlike a holonomic base's native [x, y, theta] joints, so we project it down to the
    world-frame yaw for comparison against the [x, y, theta] waypoints the A* nav planner uses.
    """
    assert move_group_ids == ["base"], f"Expected ['base'], got {move_group_ids}"
    assert len(target_pose) == 3, f"Expected [x, y, theta] pose, got {target_pose}"
    pose = self.base.pose
    theta = R.from_matrix(pose[:3, :3]).as_euler("xyz")[2]
    x_delta = pose[0, 3] - target_pose[0]
    y_delta = pose[1, 3] - target_pose[1]
    theta_delta = normalize_ang_error(theta - target_pose[2])
    return float(np.linalg.norm(np.array([x_delta, y_delta, theta_delta])))

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Exclude legs/waist, but not the pelvis, from generic arm-reach IK. With both unlocked as redundant DOF the solver wandered the leg joints and failed every grasp; a small pelvis shift is a sensible way to extend reach, bipedal leg motion is not.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Exclude legs/waist, but not the pelvis, from generic arm-reach IK.
    With both unlocked as redundant DOF the solver wandered the leg joints
    and failed every grasp; a small pelvis shift is a sensible way to
    extend reach, bipedal leg motion is not."""
    return ["legs_waist"]

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

is_close_to

is_close_to(move_group_ids: list[str], target_pose: list, threshold: float = 0.1) -> bool

Is the planar base pose within threshold of the target. The default governs "waypoint reached" for G1 nav; 0.1 because the WBC's tracking plateaus around 0.07, so 0.05 was never reached.

Source code in molmo_spaces/robots/robot_views/g1_view.py

def is_close_to(
    self, move_group_ids: list[str], target_pose: list, threshold: float = 0.1
) -> bool:
    """Is the planar base pose within `threshold` of the target. The default
    governs "waypoint reached" for G1 nav; 0.1 because the WBC's tracking
    plateaus around 0.07, so 0.05 was never reached."""
    return self.distance_to(move_group_ids, target_pose) < threshold

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

i2rt_yam_view

Implementation of the i2rt YAM robot model.

The YAM is a 6-DOF arm robot with a parallel gripper that has coupled fingers.

Each component is implemented as a MoveGroup, with the overall robot structure managed by the I2rtYamRobotView class.

Classes:

Name	Description
I2rtYamArmGroup	6-DOF arm group for the YAM robot.
I2rtYamBaseGroup	Base group for the YAM robot using mocap body control.
I2rtYamGripperGroup	Parallel gripper group for the YAM robot.
I2rtYamRobotView	Robot view for the i2rt YAM 6-DOF arm with parallel gripper.

I2rtYamArmGroup

I2rtYamArmGroup(mj_data: MjData, base_group: I2rtYamBaseGroup, namespace: str = '', grasp_site_name: str = 'grasp_site')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

6-DOF arm group for the YAM robot.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(
    self,
    mj_data: MjData,
    base_group: I2rtYamBaseGroup,
    namespace: str = "",
    grasp_site_name: str = "grasp_site",
) -> None:
    model = mj_data.model
    self._namespace = namespace
    # 6 arm joints: joint1 through joint6
    joint_ids = [model.joint(f"{namespace}joint{i + 1}").id for i in range(6)]
    # 6 arm actuators with the same names
    act_ids = [model.actuator(f"{namespace}joint{i + 1}").id for i in range(6)]
    self._arm_root_id = model.body(f"{namespace}arm").id
    self._ee_site_id = model.site(f"{namespace}{grasp_site_name}").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base_group)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

I2rtYamBaseGroup

I2rtYamBaseGroup(mj_data: MjData, namespace: str = '')

Bases: MocapRobotBaseGroup

Base group for the YAM robot using mocap body control.

Note: The mocap body is created by add_robot_to_scene and named 'base', not 'arm'. The 'arm' body from the XML is attached under this mocap body.

Initialize a MocapRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the simulation state	required
robot_base_body_id	int	The ID of the mocap body that represents the robot base. Not the Mocap ID!	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    body_id: int = mj_data.model.body(f"{namespace}base").id
    super().__init__(mj_data, body_id)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    return np.zeros((6, self.mj_model.nv))

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

I2rtYamGripperGroup

I2rtYamGripperGroup(mj_data: MjData, base_group: I2rtYamBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

Parallel gripper group for the YAM robot.

The YAM gripper has two coupled fingers (left_finger and right_finger) controlled by a single actuator. The fingers move in opposite directions due to an equality constraint: right_finger = -left_finger.

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set gripper to fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	Distance between fingers.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(self, mj_data: MjData, base_group: I2rtYamBaseGroup, namespace: str = "") -> None:
    model = mj_data.model
    self._namespace = namespace
    # Two coupled finger joints
    joint_ids = [
        model.joint(f"{namespace}left_finger").id,
        model.joint(f"{namespace}right_finger").id,
    ]
    # Single gripper actuator controls left_finger (right follows via equality constraint)
    act_ids = [model.actuator(f"{namespace}gripper").id]
    root_body_id = model.body(f"{namespace}link_6").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

Distance between fingers.

Since fingers are coupled in opposite directions (right = -left), the total opening is the sum of absolute positions.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers.

From XML: - left_finger range: -0.00205 to 0.037524 - right_finger range: -0.037524 to 0.00205 (mirrored) - Closed (both at 0): dist = 0 - Open (left at 0.037524, right at -0.037524): dist ~= 0.075

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set gripper to fully open or closed.

From XML: gripper actuator ctrlrange is 0.0 to 0.041 - 0.0 = closed - 0.041 = open

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    """Set gripper to fully open or closed.

    From XML: gripper actuator ctrlrange is 0.0 to 0.041
    - 0.0 = closed
    - 0.041 = open
    """
    self.ctrl = [0.041 if open else 0.0]

I2rtYamRobotView

I2rtYamRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Robot view for the i2rt YAM 6-DOF arm with parallel gripper.

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	I2rtYamBaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/i2rt_yam_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = I2rtYamBaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": I2rtYamArmGroup(mj_data, base, namespace=namespace),
        "gripper": I2rtYamGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: I2rtYamBaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

mobile_franka_droid_view

Classes:

Name	Description
MobileFrankaDroidBaseGroup
MobileFrankaDroidRobotView

MobileFrankaDroidBaseGroup

MobileFrankaDroidBaseGroup(mj_data: MjData, namespace: str = '')

Bases: HoloJointsRobotBaseGroup

Initialize a HoloJointsRobotBase that has virtual holonomic joints and uses site control.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
world_site_id	int	The ID of the world site	required
holo_base_site_id	int	The ID of the site that represents the holonomic base pose	required
joint_ids	list[int]	List of joint IDs that belong to the virtual holonomic base pose. NOTE: Assumed order is [x, y, theta].	required
actuator_ids	list[int]	List of actuator IDs that control the base	required
root_body_id	int	The ID of the body that represents the robot base	required

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the holonomic base actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/mobile_franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    model = mj_data.model
    world_site_id = model.site(f"{namespace}world").id
    holo_base_site_id = model.site(f"{namespace}base_site").id
    joints = [model.joint(f"{namespace}base_{axis}").id for axis in ["x", "y", "theta"]]
    act = [model.actuator(f"{namespace}base_{axis}_act").id for axis in ["x", "y", "theta"]]
    root_body_id = model.body(f"{namespace}base").id
    super().__init__(mj_data, world_site_id, holo_base_site_id, joints, act, root_body_id)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the holonomic base actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._holo_base_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

MobileFrankaDroidRobotView

MobileFrankaDroidRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	MobileFrankaDroidBaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/mobile_franka_droid_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    self._namespace = namespace
    base = MobileFrankaDroidBaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "arm": FrankaFR3ArmGroup(
            mj_data, base, namespace=namespace, grasp_site_name="gripper/grasp_site"
        ),
        "gripper": RobotIQGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: MobileFrankaDroidBaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

rby1_view

Implementation of the RBY1 robot model.

The RBY1 is a humanoid robot with: - Two 7-DOF arms with grippers - A 6-DOF torso - A mobile base with two wheels - A 2-DOF head

Each component is implemented as a MoveGroup, with the overall robot structure managed by the RBY1 RobotView class.

Classes:

Name	Description
RBY1ArmGroup	Implementation of the RBY1's 7-DOF arm, excluding the gripper.
RBY1BaseGroup	Implementation of the RBY1's mobile base.
RBY1GripperGroup	Implementation of the RBY1's gripper.
RBY1HeadGroup	Implementation of the RBY1's head.
RBY1HoloBaseGroup	Implementation of a RBY1 mobile base with virtual holonomic joints and site control.
RBY1RobotView	Implementation of the complete RBY1 robot.
RBY1TorsoGroup	Implementation of the RBY1's torso.

RBY1ArmGroup

RBY1ArmGroup(mj_data: MjData, side: Literal['left', 'right'], base: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

Implementation of the RBY1's 7-DOF arm, excluding the gripper.

Initialize an RBY1 arm.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
side	Literal['left', 'right']	Which side of the robot this arm is on ("left" or "right")	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
side
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base: RobotBaseGroup,
    namespace: str = "",
) -> None:
    """Initialize an RBY1 arm.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        side: Which side of the robot this arm is on ("left" or "right")
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    self.side = side
    self._namespace = namespace
    joint_ids = [model.joint(f"{namespace}{side}_arm_{i}").id for i in range(7)]
    act_ids = [model.actuator(f"{namespace}{side}_arm_{i + 1}_act").id for i in range(7)]
    self._ee_site_id = model.site(f"{namespace}ee_site_{side[0]}").id
    self._arm_root_id = model.body(f"{namespace}link_{side}_arm_0").id
    super().__init__(mj_data, joint_ids, act_ids, self._arm_root_id, base)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

side instance-attribute

side = side

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1BaseGroup

RBY1BaseGroup(mj_data: MjData, namespace: str = '')

Bases: FreeJointRobotBaseGroup

Implementation of the RBY1's mobile base.

The RBY1 base uses a free joint for its pose and has two wheels for mobility. The wheels are controlled independently, allowing for differential drive motion.

Initialize the RBY1 base.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
floating		Whether the base is floating.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    """Initialize the RBY1 base.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    base_joint_id = model.joint(f"{namespace}freejoint").id
    joints = [model.joint(f"{namespace}{side}_wheel").id for side in ["left", "right"]]
    act = [model.actuator(f"{namespace}{side}_wheel_act").id for side in ["left", "right"]]
    super().__init__(mj_data, base_joint_id, joints, act)

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

floating property

floating

Whether the base is floating.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1GripperGroup

RBY1GripperGroup(mj_data: MjData, side: Literal['left', 'right'], base: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

Implementation of the RBY1's gripper.

The RBY1 gripper has 2 fingers that are mechanically coupled, allowing for open/close motion of the two fingers.

Initialize an RBY1 gripper.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
side	Literal['left', 'right']	Which side of the robot this gripper is on ("left" or "right")	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the gripper actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(
    self,
    mj_data: MjData,
    side: Literal["left", "right"],
    base: RobotBaseGroup,
    namespace: str = "",
) -> None:
    """Initialize an RBY1 gripper.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        side: Which side of the robot this gripper is on ("left" or "right")
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    self._side = side
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}gripper_finger_{side[0]}{i + 1}").id for i in range(2)
    ]
    act_ids = [model.actuator(f"{namespace}{side}_finger_act").id]
    root_body_id = model.body(f"{namespace}EE_BODY_{side[0].upper()}").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base)
    self._ee_site_id = model.site(f"{namespace}ee_site_{side[0]}").id

ctrl property writable

ctrl: ndarray

Current control signals for the gripper actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool) -> None

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def set_gripper_ctrl_open(self, open: bool) -> None:
    self.ctrl = np.array([-0.05 if open else 0.0])

RBY1HeadGroup

RBY1HeadGroup(mj_data: MjData, base: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

Implementation of the RBY1's head.

The RBY1 head has 2 degrees of freedom, allowing for pan and tilt motion to look around the environment.

Initialize the RBY1 head.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, base: RobotBaseGroup, namespace: str = "") -> None:
    """Initialize the RBY1 head.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    joint_ids = [model.joint(f"{namespace}head_{i}").id for i in range(2)]
    act_ids = [model.actuator(f"{namespace}head_{i}_act").id for i in range(2)]
    root_body_id = model.body(f"{namespace}link_head_1").id
    super().__init__(mj_data, joint_ids, act_ids, root_body_id, base)
    self._head_root_id = model.body(f"{namespace}link_head_1").id
    self._head_leaf_id = model.body(f"{namespace}link_head_2").id

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1HoloBaseGroup

RBY1HoloBaseGroup(mj_data: MjData, namespace: str = '')

Bases: HoloJointsRobotBaseGroup

Implementation of a RBY1 mobile base with virtual holonomic joints and site control.

The RBY1 base uses three virtual holonomic joints for x, y and theta control.

Initialize the RBY1 holo base.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the holonomic base actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
is_mobile	bool	Whether this base is mobile (has actuators).
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, namespace: str = "") -> None:
    """Initialize the RBY1 holo base.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    world_site_id = model.site(f"{namespace}world").id
    holo_base_site_id = model.site(f"{namespace}base_site").id
    joints = [model.joint(f"{namespace}base_{axis}").id for axis in ["x", "y", "theta"]]
    act = [model.actuator(f"{namespace}base_{axis}_act").id for axis in ["x", "y", "theta"]]
    root_body_id = model.body(f"{namespace}base").id
    super().__init__(mj_data, world_site_id, holo_base_site_id, joints, act, root_body_id)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the holonomic base actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

is_mobile cached property

is_mobile: bool

Whether this base is mobile (has actuators).

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self._holo_base_site_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

RBY1RobotView

RBY1RobotView(mj_data: MjData, namespace: str = '', holo_base: bool = False)

Bases: RobotView

Implementation of the complete RBY1 robot.

The RBY1 is a humanoid robot with: - Two 7-DOF arms with grippers - A 6-DOF torso - A mobile base with two wheels (or three holonomic joints) - A 2-DOF head

Each component is implemented as a MoveGroup, with the overall robot structure managed by this class.

Initialize the RBY1 robot.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
distance_to	Calculate the distance between the current joint positions of the move groups and the target pose
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_joint_position	Get the current joint positions of the move groups
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
is_close_to	Check if the current joint positions of the move groups are close to the target pose
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base		The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name	str	The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, namespace: str = "", holo_base: bool = False) -> None:
    """Initialize the RBY1 robot.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    self._namespace = namespace
    base = (
        RBY1BaseGroup(mj_data, namespace=namespace)
        if not holo_base
        else RBY1HoloBaseGroup(mj_data, namespace=namespace)
    )
    move_groups = {
        "base": base,
        "torso": RBY1TorsoGroup(mj_data, base, namespace=namespace),
        "left_arm": RBY1ArmGroup(mj_data, "left", base, namespace=namespace),
        "right_arm": RBY1ArmGroup(mj_data, "right", base, namespace=namespace),
        "left_gripper": RBY1GripperGroup(mj_data, "left", base, namespace=namespace),
        "right_gripper": RBY1GripperGroup(mj_data, "right", base, namespace=namespace),
        "head": RBY1HeadGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(
        mj_data,
        move_groups,
    )

base property

base

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name: str

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

distance_to

distance_to(move_group_ids: list[str], target_pose: list) -> float

Calculate the distance between the current joint positions of the move groups and the target pose

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def distance_to(self, move_group_ids: list[str], target_pose: list) -> float:
    """Calculate the distance between the current joint positions of the move groups and the target pose"""
    assert len(target_pose) == 3, f"Expected [x, y, theta] pose, got {target_pose}"
    current_joint_pos = self.get_joint_position(move_group_ids)
    x_delta = current_joint_pos[0] - target_pose[0]
    y_delta = current_joint_pos[1] - target_pose[1]
    theta_delta = normalize_ang_error(current_joint_pos[2] - target_pose[2])
    return float(np.linalg.norm(np.array([x_delta, y_delta, theta_delta])))

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_joint_position

get_joint_position(move_group_ids: list[str]) -> ndarray

Get the current joint positions of the move groups

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def get_joint_position(self, move_group_ids: list[str]) -> np.ndarray:
    """Get the current joint positions of the move groups"""
    return np.concatenate(
        [
            self.get_move_group(move_group_id).joint_pos.copy()
            for move_group_id in move_group_ids
        ]
    )

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

is_close_to

is_close_to(move_group_ids: list[str], target_pose: list, threshold: float = 0.05) -> bool

Check if the current joint positions of the move groups are close to the target pose

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def is_close_to(
    self, move_group_ids: list[str], target_pose: list, threshold: float = 0.05
) -> bool:
    """Check if the current joint positions of the move groups are close to the target pose"""
    return self.distance_to(move_group_ids, target_pose) < threshold

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

RBY1TorsoGroup

RBY1TorsoGroup(mj_data: MjData, base: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, SimplyActuatedMoveGroup

Implementation of the RBY1's torso.

The RBY1 torso has 6 degrees of freedom, allowing for full control of the upper body's position and orientation relative to the base.

Initialize the RBY1 torso.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
namespace	str	Optional prefix for all joint/body names to support multiple robots	''

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
actuator_ids
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
joint_ids
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_posadr
joint_vel	ndarray	Current joint velocities.
joint_veladr
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rby1_view.py

def __init__(self, mj_data: MjData, base: RobotBaseGroup, namespace: str = "") -> None:
    """Initialize the RBY1 torso.

    Args:
        mj_data: The MuJoCo data structure containing the current simulation state
        namespace: Optional prefix for all joint/body names to support multiple robots
    """
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [model.joint(f"{namespace}torso_{i}").id for i in range(6)]
    act_ids = [model.actuator(f"{namespace}link{i + 1}_act").id for i in range(6)]
    self._torso_root_id = model.body(f"{namespace}link_torso_0").id
    self._torso_leaf_id = model.body(f"{namespace}link_torso_5").id
    super().__init__(mj_data, joint_ids, act_ids, self._torso_root_id, base)

actuator_ids property

actuator_ids

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

joint_ids property

joint_ids

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_posadr property

joint_posadr

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

joint_veladr property

joint_veladr

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

rum_gripper_view

Classes:

Name	Description
FloatingRUMBaseGroup	A robot base group for the floating RUM gripper.
FloatingRUMRobotView
RUMGripperGroup

Functions:

Name	Description
main

FloatingRUMBaseGroup

FloatingRUMBaseGroup(mj_data: MjData, namespace: str = '')

Bases: FreeJointRobotBaseGroup

A robot base group for the floating RUM gripper. The gripper is controlled via a target pose mocap body. Since this doesn't correspond to an actuator, we "fake" the actuators by overriding the appropriate methods and properties.

Initialize a FreeJointRobotBase.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
base_joint_id	int	The ID of the free joint that represents the base pose	required
joint_ids	list[int]	List of additional joint IDs that belong to this base	required
actuator_ids	list[int]	List of actuator IDs that control the base	required
floating		Whether the base is floating (concretely, whether IK should not constrain it to be on the ground)	False

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.

Attributes:

Name	Type	Description
ctrl		Current control signals for the actuators.
ctrl_limits		Control limits (min, max) for each actuator.
floating		Whether the base is floating.
is_mobile		Whether this base is mobile (has actuators).
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
mj_data
mj_model
n_actuators		Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl		Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
pose	ndarray	Get the pose of the robot base relative to the world frame.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def __init__(self, mj_data: MjData, namespace: str = ""):
    model = mj_data.model
    base_joint_id = model.joint(f"{namespace}rum_free_joint").id
    self._target_pose_body = create_mlspaces_body(mj_data, f"{namespace}target_ee_pose")
    super().__init__(mj_data, base_joint_id, [], [], floating=True)

ctrl property writable

ctrl

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits

Control limits (min, max) for each actuator.

floating property

floating

Whether the base is floating.

is_mobile cached property

is_mobile

Whether this base is mobile (has actuators).

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

pose property writable

pose: ndarray

Get the pose of the robot base relative to the world frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from world to base frame.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    body_id = self.mj_model.jnt_bodyid[self._base_joint_id]
    J = np.zeros((6, self.mj_model.nv))
    mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], body_id)
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

FloatingRUMRobotView

FloatingRUMRobotView(mj_data: MjData, namespace: str = '')

Bases: RobotView

Initialize a RobotView.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
move_groups	dict[str, MoveGroup]	Dictionary mapping move group names to their implementations	required

Methods:

Name	Description
distance_to	Calculate the planar (x, y, theta) distance from the base's current pose to a target pose.
get_ctrl_dict
get_gripper	Get a gripper by its ID.
get_gripper_movegroup_ids	Get the IDs of all gripper move groups in this robot.
get_ik_excluded_movegroup_ids	Move groups that generic IK-based manipulation policies should never
get_jacobian	Calculate the Jacobian of a move group with respect to specific input move groups.
get_move_group	Get a move group by its ID.
get_noop_ctrl_dict
get_qpos_dict	Get the joint positions of all move groups.
get_qvel_dict	Get the joint velocities of all move groups.
is_close_to	Check if the current planar base pose is close to the target pose
move_group_ids	Get the IDs of all move groups in this robot.
set_qpos_dict	Set the joint positions of all move groups.

Attributes:

Name	Type	Description
base	FloatingRUMBaseGroup	The base of the robot.
mj_data
mj_model
n_grippers	int	Number of grippers in this robot.
name		The name of this robot model.
root_body_id	int	The ID of the root body of the robot's kinematic tree.

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def __init__(self, mj_data: MjData, namespace: str = ""):
    self._namespace = namespace
    base = FloatingRUMBaseGroup(mj_data, namespace=namespace)
    move_groups = {
        "base": base,
        "gripper": RUMGripperGroup(mj_data, base, namespace=namespace),
    }
    super().__init__(mj_data, move_groups)

base property

base: FloatingRUMBaseGroup

The base of the robot.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_grippers cached property

n_grippers: int

Number of grippers in this robot.

name property

name

The name of this robot model.

root_body_id cached property

root_body_id: int

The ID of the root body of the robot's kinematic tree.

This is not necessarily base.root_body_id: the base move group's root body is the base itself, which may be nested under an outer wrapper body (RBY1's MJCF nests robot_0/base under robot_0/). Contacts identify a body's tree via model.body_rootid, so comparisons against contact bodies must use this.

distance_to

distance_to(move_group_ids: list[str], target_pose: list) -> float

Calculate the planar (x, y, theta) distance from the base's current pose to a target pose.

The floating base's pose is a full 6-DOF transform (see FreeJointRobotBaseGroup.pose), unlike a holonomic base's native [x, y, theta] joints, so we project it down to the world-frame yaw for comparison against the [x, y, theta] waypoints the A* nav planner uses.

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def distance_to(self, move_group_ids: list[str], target_pose: list) -> float:
    """Calculate the planar (x, y, theta) distance from the base's current pose to a target pose.

    The floating base's pose is a full 6-DOF transform (see FreeJointRobotBaseGroup.pose),
    unlike a holonomic base's native [x, y, theta] joints, so we project it down to the
    world-frame yaw for comparison against the [x, y, theta] waypoints the A* nav planner uses.
    """
    assert move_group_ids == ["base"], f"Expected ['base'], got {move_group_ids}"
    assert len(target_pose) == 3, f"Expected [x, y, theta] pose, got {target_pose}"
    pose = self.base.pose
    theta = R.from_matrix(pose[:3, :3]).as_euler("xyz")[2]
    x_delta = pose[0, 3] - target_pose[0]
    y_delta = pose[1, 3] - target_pose[1]
    theta_delta = normalize_ang_error(theta - target_pose[2])
    return float(np.linalg.norm(np.array([x_delta, y_delta, theta_delta])))

get_ctrl_dict

get_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].ctrl for mg_id in move_group_ids}

get_gripper

get_gripper(gripper_group_id: str)

Get a gripper by its ID.

Parameters:

Name	Type	Description	Default
gripper_group_id	str	The ID of the gripper group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper(self, gripper_group_id: str):
    """Get a gripper by its ID.

    Args:
        gripper_group_id: The ID of the gripper group to get
    """
    gripper = self.get_move_group(gripper_group_id)
    assert isinstance(gripper, GripperGroup)
    return gripper

get_gripper_movegroup_ids

get_gripper_movegroup_ids() -> list[str]

Get the IDs of all gripper move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_gripper_movegroup_ids(self) -> list[str]:
    """Get the IDs of all gripper move groups in this robot."""
    if hasattr(self, "_gripper_movegroup_ids_cache"):
        return self._gripper_movegroup_ids_cache
    self._gripper_movegroup_ids_cache = [
        mg_id
        for mg_id in self.move_group_ids()
        if isinstance(self._move_groups[mg_id], GripperGroup)
    ]
    return self._gripper_movegroup_ids_cache

get_ik_excluded_movegroup_ids

get_ik_excluded_movegroup_ids() -> list[str]

Move groups that generic IK-based manipulation policies should never unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

Default empty: existing robots (RBY1's holonomic base, Franka's fixed mount, etc.) have few enough non-arm DOF that letting a manipulation policy's IK solver treat "every non-gripper move group" as free to help reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn) is reasonable. Override for robots where that assumption breaks down -- e.g. a bipedal humanoid's legs, which have no business being moved by an arm-reach IK solve.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_ik_excluded_movegroup_ids(self) -> list[str]:
    """Move groups that generic IK-based manipulation policies should never
    unlock or overwrite, on top of grippers (see get_gripper_movegroup_ids()).

    Default empty: existing robots (RBY1's holonomic base, Franka's fixed
    mount, etc.) have few enough non-arm DOF that letting a manipulation
    policy's IK solver treat "every non-gripper move group" as free to help
    reach a target (see base_object_manipulation_planner_policy._tcp_to_jp_fn)
    is reasonable. Override for robots where that assumption breaks down --
    e.g. a bipedal humanoid's legs, which have no business being moved by an
    arm-reach IK solve.
    """
    return []

get_jacobian

get_jacobian(move_group_id: str, input_move_group_ids: list[str]) -> ndarray

Calculate the Jacobian of a move group with respect to specific input move groups.

This allows computing the Jacobian while locking certain joints (by excluding their move groups from input_move_group_ids).

Parameters:

Name	Type	Description	Default
move_group_id	str	The ID of the move group to get the jacobian of	required
input_move_group_ids	list[str]	The IDs of the move groups to use as input	required

Returns:

Type	Description
ndarray	The (6, N) jacobian of the move group, where N is the total number of degrees
ndarray	of freedom of the input move groups.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self, move_group_id: str, input_move_group_ids: list[str]) -> np.ndarray:
    """Calculate the Jacobian of a move group with respect to specific input move groups.

    This allows computing the Jacobian while locking certain joints (by excluding their
    move groups from input_move_group_ids).

    Args:
        move_group_id: The ID of the move group to get the jacobian of
        input_move_group_ids: The IDs of the move groups to use as input

    Returns:
        The (6, N) jacobian of the move group, where N is the total number of degrees
        of freedom of the input move groups.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    J = self._move_groups[move_group_id].get_jacobian()
    qveladr: list[int] = []
    for mg_id in input_move_group_ids:
        mg = self._move_groups[mg_id]
        qveladr.extend(mg._joint_veladr)

        # don't allow non-floating robot bases to move vertically, or pitch/roll
        if isinstance(mg, FreeJointRobotBaseGroup) and not mg.floating:
            for jnt_id in mg._joint_ids:
                if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
                    dofadrs = self.mj_model.jnt_dofadr[jnt_id] + np.array([2, 3, 4])
                    J[:, dofadrs] = 0.0
    J = J[:, qveladr]
    return J

get_move_group

get_move_group(mg_id: str)

Get a move group by its ID.

Parameters:

Name	Type	Description	Default
mg_id	str	The ID of the move group to get	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_move_group(self, mg_id: str):
    """Get a move group by its ID.

    Args:
        mg_id: The ID of the move group to get
    """
    return self._move_groups[mg_id]

get_noop_ctrl_dict

get_noop_ctrl_dict(move_group_ids: list[str] | None = None)

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_noop_ctrl_dict(self, move_group_ids: list[str] | None = None):
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].noop_ctrl for mg_id in move_group_ids}

get_qpos_dict

get_qpos_dict(move_group_ids: list[str] | None = None)

Get the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint positions of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint positions.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qpos_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint positions of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint positions of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint positions.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_pos for mg_id in move_group_ids}

get_qvel_dict

get_qvel_dict(move_group_ids: list[str] | None = None)

Get the joint velocities of all move groups.

Parameters:

Name	Type	Description	Default
move_group_ids	list[str] | None	The IDs of the move groups to get the joint velocities of. If None, all move groups will be included.	None

Returns:

Type	Description
	A dictionary mapping move group IDs to their joint velocities.

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_qvel_dict(self, move_group_ids: list[str] | None = None):
    """Get the joint velocities of all move groups.

    Args:
        move_group_ids: The IDs of the move groups to get the joint velocities of.
                        If None, all move groups will be included.

    Returns:
        A dictionary mapping move group IDs to their joint velocities.
    """
    if move_group_ids is None:
        move_group_ids = self.move_group_ids()
    return {mg_id: self._move_groups[mg_id].joint_vel for mg_id in move_group_ids}

is_close_to

is_close_to(move_group_ids: list[str], target_pose: list, threshold: float = 0.05) -> bool

Check if the current planar base pose is close to the target pose

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def is_close_to(
    self, move_group_ids: list[str], target_pose: list, threshold: float = 0.05
) -> bool:
    """Check if the current planar base pose is close to the target pose"""
    return self.distance_to(move_group_ids, target_pose) < threshold

move_group_ids

move_group_ids() -> list[str]

Get the IDs of all move groups in this robot.

Source code in molmo_spaces/robots/robot_views/abstract.py

def move_group_ids(self) -> list[str]:
    """Get the IDs of all move groups in this robot."""
    return list(self._move_groups.keys())

set_qpos_dict

set_qpos_dict(qpos_dict: dict[str, ndarray]) -> None

Set the joint positions of all move groups.

Parameters:

Name	Type	Description	Default
qpos_dict	dict[str, ndarray]	A dictionary mapping move group IDs to their joint positions.	required

Source code in molmo_spaces/robots/robot_views/abstract.py

def set_qpos_dict(self, qpos_dict: dict[str, np.ndarray]) -> None:
    """Set the joint positions of all move groups.

    Args:
        qpos_dict: A dictionary mapping move group IDs to their joint positions.
    """
    for mg_id, qpos in qpos_dict.items():
        self._move_groups[mg_id].joint_pos = qpos

RUMGripperGroup

RUMGripperGroup(mj_data: MjData, base_group: RobotBaseGroup, namespace: str = '')

Bases: MJCFFrameMixin, GripperGroup

Initialize a MoveGroup.

Parameters:

Name	Type	Description	Default
mj_data	MjData	The MuJoCo data structure containing the current simulation state	required
joint_ids	list[int]	List of joint IDs that belong to this move group	required
actuator_ids	list[int]	List of actuator IDs that control the joints	required
root_body_id	int	The ID of the root body of this move group	required
robot_base_group	RobotBaseGroup | None	The RobotBaseGroup for the robot. If None, this MoveGroup is assumed to be the base.	None

Methods:

Name	Description
get_jacobian	Returns the (6, model.nv) jacobian of the move group's leaf frame.
integrate_joint_vel	Integrate joint velocities by 1 unit time to get joint positions.
set_gripper_ctrl_open	Set the gripper commanded position to be fully open or closed.

Attributes:

Name	Type	Description
ctrl	ndarray	Current control signals for the actuators.
ctrl_limits	ndarray	Control limits (min, max) for each actuator.
inter_finger_dist	float	The distance between the two fingers of the gripper.
inter_finger_dist_range	tuple[float, float]	The (min, max) of the distance between the two fingers of the gripper.
is_open	bool	Whether the gripper is open.
joint_pos	ndarray	Current joint positions.
joint_pos_limits	ndarray	Joint position limits (min, max) for each joint.
joint_vel	ndarray	Current joint velocities.
leaf_frame_id	int	The ID of the leaf frame, either a body ID or a site ID.
leaf_frame_to_robot	ndarray	Returns the pose of the leaf frame relative to the robot frame.
leaf_frame_to_root	ndarray	Returns the pose of the leaf frame relative to the root frame.
leaf_frame_to_world	ndarray
leaf_frame_type		The type of the leaf frame.
mj_data
mj_model
n_actuators	int	Number of actuators in this move group.
n_joints	int	Number of joints in this move group.
name	str	The name of this move group, which matches the move group ID in the containing robot view.
noop_ctrl	ndarray	Get a control signal that maintains the current state.
pos_dim	int	Dimension of the ambient space of the manifold of joint positions.
root_body_id	int	The ID of the root body of this move group.
root_frame_to_robot	ndarray	Returns the pose of the root frame relative to the robot frame.
root_frame_to_world	ndarray
vel_dim	int	Dimension of the space of joint velocities.

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def __init__(self, mj_data: MjData, base_group: RobotBaseGroup, namespace: str = ""):
    model = mj_data.model
    self._namespace = namespace
    joint_ids = [
        model.joint(f"{namespace}finger_left_joint").id,
        model.joint(f"{namespace}finger_right_joint").id,
    ]
    actuator_ids = [model.actuator(f"{namespace}fingers_actuator").id]
    super().__init__(mj_data, joint_ids, actuator_ids, base_group.root_body_id, base_group)
    self._ee_site_id = model.site(f"{namespace}grasp_site").id
    self._left_fingertip_geom_id = model.geom(f"{namespace}left_fingertip").id
    self._right_fingertip_geom_id = model.geom(f"{namespace}right_fingertip").id

ctrl property writable

ctrl: ndarray

Current control signals for the actuators.

ctrl_limits cached property

ctrl_limits: ndarray

Control limits (min, max) for each actuator.

inter_finger_dist property

inter_finger_dist: float

The distance between the two fingers of the gripper.

inter_finger_dist_range property

inter_finger_dist_range: tuple[float, float]

The (min, max) of the distance between the two fingers of the gripper.

is_open property

is_open: bool

Whether the gripper is open.

joint_pos property writable

joint_pos: ndarray

Current joint positions.

joint_pos_limits property

joint_pos_limits: ndarray

Joint position limits (min, max) for each joint.

joint_vel property writable

joint_vel: ndarray

Current joint velocities.

leaf_frame_id property

leaf_frame_id: int

The ID of the leaf frame, either a body ID or a site ID.

leaf_frame_to_robot property

leaf_frame_to_robot: ndarray

Returns the pose of the leaf frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to leaf frame.

leaf_frame_to_root property

leaf_frame_to_root: ndarray

Returns the pose of the leaf frame relative to the root frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from root to leaf frame.

leaf_frame_to_world property

leaf_frame_to_world: ndarray

leaf_frame_type property

leaf_frame_type

The type of the leaf frame.

mj_data instance-attribute

mj_data = mj_data

mj_model instance-attribute

mj_model = mj_data.model

n_actuators cached property

n_actuators: int

Number of actuators in this move group.

n_joints cached property

n_joints: int

Number of joints in this move group.

name property

name: str

The name of this move group, which matches the move group ID in the containing robot view. Not safe to call before the move group is added to a robot view.

noop_ctrl property

noop_ctrl: ndarray

Get a control signal that maintains the current state.

pos_dim cached property

pos_dim: int

Dimension of the ambient space of the manifold of joint positions. This may be different from the number of joints, in the case of free or ball joints.

root_body_id property

root_body_id: int

The ID of the root body of this move group.

root_frame_to_robot property

root_frame_to_robot: ndarray

Returns the pose of the root frame relative to the robot frame.

Returns:

Type	Description
ndarray	A 4x4 numpy array representing the rigid transformation from robot to root frame.

root_frame_to_world property

root_frame_to_world: ndarray

vel_dim cached property

vel_dim: int

Dimension of the space of joint velocities. This may be different from the number of position dimensions, in the case of free or ball joints.

get_jacobian

get_jacobian() -> ndarray

Returns the (6, model.nv) jacobian of the move group's leaf frame.

The jacobian maps joint velocities to the spatial velocity of the leaf frame.

Returns:

Type	Description
ndarray	A 6xN numpy array where N is the number of degrees of freedom in the model.

See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac

Source code in molmo_spaces/robots/robot_views/abstract.py

def get_jacobian(self) -> np.ndarray:
    """
    Returns the (6, model.nv) jacobian of the move group's leaf frame.

    The jacobian maps joint velocities to the spatial velocity of the leaf frame.

    Returns:
        A 6xN numpy array where N is the number of degrees of freedom in the model.

    See: https://mujoco.readthedocs.io/en/stable/APIreference/APIfunctions.html#mj-jac
    """
    assert isinstance(self, MoveGroup), (
        f"{self.__class__.__name__} must be used with a MoveGroup"
    )

    J = np.zeros((6, self.mj_model.nv))
    if self.leaf_frame_type == "site":
        mj.mj_jacSite(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    elif self.leaf_frame_type == "body":
        mj.mj_jacBody(self.mj_model, self.mj_data, J[:3], J[3:], self.leaf_frame_id)
    else:
        raise ValueError(f"Invalid leaf frame type: {self.leaf_frame_type}")
    return J

integrate_joint_vel

integrate_joint_vel(joint_pos: ndarray, joint_vel: ndarray) -> ndarray

Integrate joint velocities by 1 unit time to get joint positions. This does not modify the state.

Parameters:

Name	Type	Description	Default
joint_pos	ndarray	Joint positions at the start of the integration	required
joint_vel	ndarray	Joint velocities to integrate	required

Returns:

Type	Description
ndarray	Joint positions at the end of the integration

Source code in molmo_spaces/robots/robot_views/abstract.py

def integrate_joint_vel(self, joint_pos: np.ndarray, joint_vel: np.ndarray) -> np.ndarray:
    """
    Integrate joint velocities by 1 unit time to get joint positions.
    This does not modify the state.

    Args:
        joint_pos: Joint positions at the start of the integration
        joint_vel: Joint velocities to integrate

    Returns:
        Joint positions at the end of the integration
    """
    new_jp = joint_pos.copy()
    i = 0
    j = 0
    for jnt_id in self._joint_ids:
        if self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_FREE:
            trf = pos_quat_to_pose_mat(self.joint_pos[i : i + 3], self.joint_pos[i + 3 : i + 7])
            twist = joint_vel[j : j + 6]
            delta_trf = twist_to_transform(twist[:3], twist[3:])
            trf = trf @ delta_trf
            new_jp[i : i + 3], new_jp[i + 3 : i + 7] = pose_mat_to_pos_quat(trf)
            i += 7
            j += 6
        elif self.mj_model.jnt_type[jnt_id] == mj.mjtJoint.mjJNT_BALL:
            rotmat = R.from_quat(self.joint_pos[i : i + 4], scalar_first=True).as_matrix()
            twist = joint_vel[j : j + 3]
            delta_rotmat = R.from_rotvec(twist).as_matrix()
            rotmat = rotmat @ delta_rotmat
            new_jp[i : i + 4] = R.from_matrix(rotmat).as_quat(scalar_first=True)
            i += 4
            j += 3
        else:
            new_jp[i] += joint_vel[j]
            i += 1
            j += 1
    return new_jp

set_gripper_ctrl_open

set_gripper_ctrl_open(open: bool)

Set the gripper commanded position to be fully open or closed.

Parameters:

Name	Type	Description	Default
open	bool	True to open the gripper, False to close it	required

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def set_gripper_ctrl_open(self, open: bool):
    self.ctrl = [0 if open else -255]

main

main()

Source code in molmo_spaces/robots/robot_views/rum_gripper_view.py

def main():
    import time

    import mujoco
    import numpy as np
    from mujoco import MjData, MjModel
    from mujoco.viewer import launch_passive

    from molmo_spaces.configs.robot_configs import FloatingRUMRobotConfig

    np.set_printoptions(linewidth=np.inf)

    robot_config = FloatingRUMRobotConfig()
    xml_path = str(robot_config.get_robot_xml_path())
    model = MjModel.from_xml_path(xml_path)

    data = MjData(model)
    mujoco.mj_forward(model, data)

    ns = ""
    robot_view = FloatingRUMRobotView(data, ns)
    robot_view.base.pose = np.eye(4)
    robot_view.base.ctrl = np.array([0, 0, 0, 1.0, 0, 0, 0])

    with launch_passive(model, data) as viewer:
        while viewer.is_running():
            viewer.sync()
            x = np.sin(data.time)
            y = 1 - np.cos(data.time)
            robot_view.base.ctrl = np.array([x, y, 0, 1.0, 0, 0, 0])
            mujoco.mj_step(model, data)
            time.sleep(0.02)

utils

Modules:

Name	Description
randomization

randomization

Functions:

Name	Description
speckle_texture

speckle_texture

speckle_texture(base_color, size=256, noise_strength=0.1, num_blobs=80, blob_size_range=(5, 25), blob_variation=0.1) -> Image

Source code in molmo_spaces/robots/utils/randomization.py

def speckle_texture(
    base_color,
    size=256,
    noise_strength=0.1,
    num_blobs=80,
    blob_size_range=(5, 25),
    blob_variation=0.1,
) -> Image.Image:
    img = np.ones((size, size, 3)) * np.array(base_color)
    noise = np.random.normal(0, noise_strength, (size, size, 1))
    img += noise
    img = np.clip(img, 0, 1)
    img = (img * 255).astype(np.uint8)

    pil_img = Image.fromarray(img)
    draw = ImageDraw.Draw(pil_img)

    # Draw chunky rectangular or elliptical blobs
    for _ in range(num_blobs):
        x = np.random.randint(0, size)
        y = np.random.randint(0, size)
        w = np.random.randint(*blob_size_range)
        h = np.random.randint(*blob_size_range)

        variation = np.random.uniform(-blob_variation, blob_variation)
        blob_color = tuple(int(np.clip((c + variation) * 255, 0, 255)) for c in base_color)

        if np.random.random() > 0.5:
            draw.ellipse((x, y, x + w, y + h), fill=blob_color)
        else:
            draw.rectangle((x, y, x + w, y + h), fill=blob_color)

    return pil_img