Skip to content

configs

configs

Configuration module for MolmoSpaces experiments.

This module provides configuration classes organized by category: - abstract_config: Base Config class - abstract_exp_config: Base experiment configuration - camera_configs: Camera-related configurations - robot_configs: Robot-related configurations - task_configs: Task-related configurations - task_sampler_configs: Task sampler-related configurations - policy_configs: Policy-related configurations

Modules:

Name Description
abstract_config
abstract_exp_config
base_nav_to_obj_config

Example configuration for RBY1 navigation to object data generation using the extracted task sampler.
base_open_task_configs
base_packing_configs
base_pick_and_place_color_configs
base_pick_and_place_configs
base_pick_and_place_next_to_configs
base_pick_config

Example configuration for Franka pick-and-place data generation using the extracted task sampler.
camera_configs
dummy_config
policy_configs

Policy configuration classes for MolmoSpaces experiments.
policy_configs_baselines
robot_configs

Robot configuration classes for MolmoSpaces experiments.
task_configs

Task configuration classes for MolmoSpaces experiments.
task_sampler_configs

Task sampler configuration classes for MolmoSpaces experiments.

Classes:

Name Description
BaseMujocoTaskConfig

Base configuration for MuJoCo tasks.
BaseMujocoTaskSamplerConfig

Base configuration for task samplers.
BasePolicyConfig

Base configuration for policies.
BaseRobotConfig

Base configuration for robot setup.
CameraConfig

Base specification for a single camera.
CameraSystemConfig

Complete camera system configuration.
Config

Base configuration class that can be extended for specific configurations.
FixedExocentricCameraConfig

Fixed external camera at a specific world position.
FrankaDroidCameraSystem

Camera system for Franka with DROID-style fixed cameras.
FrankaRandomizedD405D455CameraSystem

Camera system for Franka pick-and-place tasks with wrist cam and 2 randomized exo cams.
FrankaRobotConfig

Configuration for Franka FR3 robot.
MjcfCameraConfig

Camera defined in the MJCF file.
MlSpacesExpConfig

Base configuration class for experiments.
PickTaskConfig

Configuration for Franka move-to-pose task.
PickTaskSamplerConfig

Configuration for Franka move-to-pose task sampler.
RBY1GoProD455CameraSystem

Camera system for RBY1 with GoPro head camera and D455 wrist cameras.
RBY1MjcfCameraSystem

Camera system using RBY1's built-in MJCF cameras.
RandomizedExocentricCameraConfig

Randomized external camera positioned around a workspace center.
RobotMountedCameraConfig

Camera dynamically mounted to a robot body.

__all__ module-attribute 

__all__ = ['Config', 'MlSpacesExpConfig', 'CameraSystemConfig', 'CameraConfig', 'MjcfCameraConfig', 'RobotMountedCameraConfig', 'FixedExocentricCameraConfig', 'RandomizedExocentricCameraConfig', 'RBY1MjcfCameraSystem', 'RBY1GoProD455CameraSystem', 'FrankaRandomizedD405D455CameraSystem', 'FrankaDroidCameraSystem', 'BaseRobotConfig', 'FrankaRobotConfig', 'BaseMujocoTaskConfig', 'PickTaskConfig', 'BaseMujocoTaskSamplerConfig', 'PickTaskSamplerConfig', 'BasePolicyConfig', 'ObjectManipulationPlannerPolicyConfig']

BaseMujocoTaskConfig

Bases: Config

Base configuration for MuJoCo tasks.

NOTE: If these task config parameters are left to None, they will be sampled by the task sampler. If these task config parameters are not None, their value will take precedence over any parameters sampled by the task sampler and will remain fixed across all simulation tasks sampled by the task sampler.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
model_config
object_poses dict[str, list[float]] | None
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool

action_dtype class-attribute instance-attribute 

action_dtype: str = 'float32'

added_objects class-attribute instance-attribute 

added_objects: dict[str, Path] = {}

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

object_poses class-attribute instance-attribute 

object_poses: dict[str, list[float]] | None = None

referral_expressions class-attribute instance-attribute 

referral_expressions: dict[str, str] = {}

referral_expressions_priority class-attribute instance-attribute 

referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}

robot_base_pose class-attribute instance-attribute 

robot_base_pose: list[float] | None = None

task_cls instance-attribute 

task_cls: type | None

tracked_object_names class-attribute instance-attribute 

tracked_object_names: list[str] | None = None

use_sensors class-attribute instance-attribute 

use_sensors: bool = True

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

BaseMujocoTaskSamplerConfig

Bases: Config

Base configuration for task samplers.

A task is sampled based on this configuration.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
check_robot_placement_visibility bool
enable_texture_randomization bool
episodes_per_batch int
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_tasks int | None
max_total_attempts_multiplier int
model_config
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool

check_robot_placement_visibility class-attribute instance-attribute 

check_robot_placement_visibility: bool = True

enable_texture_randomization class-attribute instance-attribute 

enable_texture_randomization: bool = False

episodes_per_batch class-attribute instance-attribute 

episodes_per_batch: int = 4

house_inds instance-attribute 

house_inds: list[int] | None

house_variant class-attribute instance-attribute 

house_variant: str = 'ceiling'

license_policy class-attribute instance-attribute 

license_policy: LicensePolicy = LicensePolicy.NONE

max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 

max_allowed_sequential_irrecoverable_failures: int = 5

max_allowed_sequential_rollout_failures class-attribute instance-attribute 

max_allowed_sequential_rollout_failures: int = 10

max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 

max_allowed_sequential_task_sampler_failures: int = 10

max_asset_failures class-attribute instance-attribute 

max_asset_failures: int = 10

max_tasks instance-attribute 

max_tasks: int | None

max_total_attempts_multiplier class-attribute instance-attribute 

max_total_attempts_multiplier: int = 6

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

occupancy_map_cache_size class-attribute instance-attribute 

occupancy_map_cache_size: int = 4

occupancy_map_impl class-attribute instance-attribute 

occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR

randomize_dynamics class-attribute instance-attribute 

randomize_dynamics: bool = False

randomize_lighting class-attribute instance-attribute 

randomize_lighting: bool = False

randomize_robot_textures class-attribute instance-attribute 

randomize_robot_textures: bool = False

randomize_textures class-attribute instance-attribute 

randomize_textures: bool = False

randomize_textures_all class-attribute instance-attribute 

randomize_textures_all: bool = False

robot_placement_exclusion_threshold class-attribute instance-attribute 

robot_placement_exclusion_threshold: float = 0.15

robot_placement_rotation_range_rad class-attribute instance-attribute 

robot_placement_rotation_range_rad: float = math.radians(45)

samples_per_house instance-attribute 

samples_per_house: int | None

scene_xml_paths class-attribute instance-attribute 

scene_xml_paths: list[str] | None = None

sim_settle_timesteps class-attribute instance-attribute 

sim_settle_timesteps: int = 500

task_batch_size instance-attribute 

task_batch_size: int

task_sampler_class class-attribute instance-attribute 

task_sampler_class: type | None = None

verbose class-attribute instance-attribute 

verbose: bool = False

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

BasePolicyConfig

Bases: Config

Base configuration for policies.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
force_enable_depth bool

Whether or not to require all cameras to record depth
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.

force_enable_depth class-attribute instance-attribute 

force_enable_depth: bool = False

Whether or not to require all cameras to record depth

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

policy_cls instance-attribute 

policy_cls: type[BasePolicy] | None

policy_factory instance-attribute 

policy_factory: PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type instance-attribute 

policy_type: str

Type of the policy, e.g., "planner", "teleop", "learned", etc.

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

BaseRobotConfig

Bases: Config

Base configuration for robot setup.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path

K_damping class-attribute instance-attribute 

K_damping: list[float] | None = None

K_stiffness class-attribute instance-attribute 

K_stiffness: list[float] | None = None

action_noise_config class-attribute instance-attribute 

action_noise_config: ActionNoiseConfig | None = None

command_mode instance-attribute 

command_mode: dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 

force_limit: list[float] | None = None

gravcomp class-attribute instance-attribute 

gravcomp: bool = False

init_qpos instance-attribute 

init_qpos: dict[str, list[float]]

init_qpos_noise_range instance-attribute 

init_qpos_noise_range: dict[str, list[float]] | None

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

name instance-attribute 

name: str | None

robot_cls instance-attribute 

robot_cls: type[Robot] | None

robot_dir class-attribute instance-attribute 

robot_dir: Path | None = None

robot_factory instance-attribute 

robot_factory: Callable[[MjData, Any], Robot] | None

robot_namespace instance-attribute 

robot_namespace: str

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory instance-attribute 

robot_view_factory: RobotViewFactory | None

robot_xml_path instance-attribute 

robot_xml_path: Path

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

get_robot_dir 

get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)

get_robot_xml_path 

get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

model_post_init 

model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
117
118
119
120
def model_post_init(self, _context):
    """Ensure action_noise_config is always initialized, even when loading from old configs."""
    if self.action_noise_config is None:
        object.__setattr__(self, "action_noise_config", ActionNoiseConfig())

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

CameraConfig

Bases: Config, ABC

Base specification for a single camera.

Each camera spec defines how one camera should be created and configured. Subclasses implement different camera types (MJCF, robot-mounted, exocentric).

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
fov float | None
is_warped bool
model_config
name str
record_depth bool
skip_erosion bool
visibility_constraints dict[str, float] | None

fov class-attribute instance-attribute 

fov: float | None = None

is_warped class-attribute instance-attribute 

is_warped: bool = False

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

name instance-attribute 

name: str

record_depth class-attribute instance-attribute 

record_depth: bool = False

skip_erosion class-attribute instance-attribute 

skip_erosion: bool = False

visibility_constraints class-attribute instance-attribute 

visibility_constraints: dict[str, float] | None = None

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

CameraSystemConfig

Bases: Config

Complete camera system configuration.

Defines all cameras that should be set up in the environment, along with shared settings like resolution.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config

cameras class-attribute instance-attribute 

cameras: list[CameraConfig] = []

img_resolution class-attribute instance-attribute 

img_resolution: tuple[int, int] = (640, 480)

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

add_camera 

add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

get_camera_by_name 

get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

Config

Bases: BaseModel

Base configuration class that can be extended for specific configurations. Provides methods to convert to dict, json, and to save/load from files.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
model_config

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FixedExocentricCameraConfig

Bases: CameraConfig

Fixed external camera at a specific world position.

Useful for consistent third-person views, overhead cameras, or monitoring positions. Can optionally add small amounts of noise for data augmentation.

TODO: should this also have a quaternion option? was figuring this would be most useful for fixed eval episodes

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
forward list[float]
fov float | None
is_warped bool
model_config
name str
orientation_noise_degrees float | Triple[float] | None
pos list[float]
pos_noise_range tuple[float, float] | tuple[Triple[float], Triple[float]] | None
record_depth bool
skip_erosion bool
up list[float]
visibility_constraints dict[str, float] | None

forward instance-attribute 

forward: list[float]

fov class-attribute instance-attribute 

fov: float | None = None

is_warped class-attribute instance-attribute 

is_warped: bool = False

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

name instance-attribute 

name: str

orientation_noise_degrees class-attribute instance-attribute 

orientation_noise_degrees: float | Triple[float] | None = None

pos instance-attribute 

pos: list[float]

pos_noise_range class-attribute instance-attribute 

pos_noise_range: tuple[float, float] | tuple[Triple[float], Triple[float]] | None = None

record_depth class-attribute instance-attribute 

record_depth: bool = False

skip_erosion class-attribute instance-attribute 

skip_erosion: bool = False

up instance-attribute 

up: list[float]

visibility_constraints class-attribute instance-attribute 

visibility_constraints: dict[str, float] | None = None

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaDroidCameraSystem

Bases: CameraSystemConfig

Camera system for Franka with DROID-style fixed cameras.

Uses wrist camera plus DROID-style exocentric camera mounted to robot base. All cameras are deterministic (no noise) for consistent, reproducible viewpoints. This matches the behavior of the old cameras_fixed_droid=True setting.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config

cameras class-attribute instance-attribute 

cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='gripper/wrist_camera', robot_namespace='robot_0/', fov=56.74), RobotMountedCameraConfig(name='exo_camera_1', reference_body_names=['robot_0/fr3_link0'], camera_offset=[0.1, 0.57, 0.66], camera_quaternion=[-0.3633, -0.1241, 0.4263, 0.8191], fov=71.0, visibility_constraints={'__task_objects__': 0.001})]

img_resolution class-attribute instance-attribute 

img_resolution: tuple[int, int] = (640, 368)

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

add_camera 

add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

get_camera_by_name 

get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaRandomizedD405D455CameraSystem

Bases: CameraSystemConfig

Camera system for Franka pick-and-place tasks with wrist cam and 2 randomized exo cams.

Uses workspace center from task sampler for dynamic placement. The task sampler should implement get_workspace_center() and resolve_visibility_object() to provide runtime information without modifying the camera config.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config

cameras class-attribute instance-attribute 

cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='wrist_cam', robot_namespace='robot_0/', fov=58.0, fov_noise_degrees=(-10.0, 10.0), pos_noise_range=(-0.015, 0.015), orientation_noise_degrees=8.0), RandomizedExocentricCameraConfig(name='exo_camera_1', distance_range=(0.2, 0.8), height_range=(0.4, 0.8), azimuth_range=(0, 2 * np.pi), fov_range=(50, 90), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, allow_relaxed_constraints=False), RandomizedExocentricCameraConfig(name='exo_camera_2', distance_range=(0.2, 0.8), height_range=(0.4, 0.8), azimuth_range=(0, 2 * np.pi), fov_range=(50, 90), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, allow_relaxed_constraints=False)]

img_resolution class-attribute instance-attribute 

img_resolution: tuple[int, int] = (640, 368)

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

add_camera 

add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

get_camera_by_name 

get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaRobotConfig

Bases: BaseRobotConfig

Configuration for Franka FR3 robot.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
base_size list[float] | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
perturb_texture_probability float
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path

K_damping class-attribute instance-attribute 

K_damping: list[float] | None = None

K_stiffness class-attribute instance-attribute 

K_stiffness: list[float] | None = None

action_noise_config class-attribute instance-attribute 

action_noise_config: ActionNoiseConfig | None = None

base_size class-attribute instance-attribute 

base_size: list[float] | None = [0.5, 0.5, 0.58]

command_mode class-attribute instance-attribute 

command_mode: dict[str, str | None] = {'arm': 'joint_position', 'gripper': 'joint_position'}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 

force_limit: list[float] | None = None

gravcomp class-attribute instance-attribute 

gravcomp: bool = True

init_qpos class-attribute instance-attribute 

init_qpos: dict[str, list[float]] = {'arm': [0, -0.7853, 0, -2.35619, 0, 1.57079, 0.0], 'gripper': [0.00296, 0.00296]}

init_qpos_noise_range class-attribute instance-attribute 

init_qpos_noise_range: dict[str, list[float]] | None = {'arm': [0.025, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175]}

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

name class-attribute instance-attribute 

name: str | None = 'franka_droid'

perturb_texture_probability class-attribute instance-attribute 

perturb_texture_probability: float = 0.7

robot_cls class-attribute instance-attribute 

robot_cls: type[Robot] | None = FrankaRobot

robot_dir class-attribute instance-attribute 

robot_dir: Path | None = None

robot_factory class-attribute instance-attribute 

robot_factory: Callable[[MjData, Any], Robot] | None = FrankaRobot

robot_namespace class-attribute instance-attribute 

robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 

robot_view_factory: RobotViewFactory | None = FrankaDroidRobotView

robot_xml_path class-attribute instance-attribute 

robot_xml_path: Path = Path('model.xml')

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

get_robot_dir 

get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)

get_robot_xml_path 

get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

model_post_init 

model_post_init(_context: Any)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
169
170
171
172
173
174
def model_post_init(self, _context: Any):
    super().model_post_init(_context)
    if "gripper" in self.command_mode:
        assert self.command_mode["gripper"] == "joint_position"
    if "arm" in self.command_mode:
        assert self.command_mode["arm"] in ["joint_position", "joint_rel_position"]

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

MjcfCameraConfig

Bases: CameraConfig

Camera defined in the MJCF file.

This references a camera that already exists in the scene MJCF or robot MJCF. Useful for cameras with fixed mounting in robot models.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
fov float | None
fov_noise_degrees tuple[float, float] | None
is_warped bool
mjcf_name str
model_config
name str
orientation_noise_degrees float | Triple[float] | None
pos_noise_range tuple[float, float] | tuple[Triple[float], Triple[float]] | None
record_depth bool
robot_namespace str | None
skip_erosion bool
visibility_constraints dict[str, float] | None

fov class-attribute instance-attribute 

fov: float | None = None

fov_noise_degrees class-attribute instance-attribute 

fov_noise_degrees: tuple[float, float] | None = None

is_warped class-attribute instance-attribute 

is_warped: bool = False

mjcf_name instance-attribute 

mjcf_name: str

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

name instance-attribute 

name: str

orientation_noise_degrees class-attribute instance-attribute 

orientation_noise_degrees: float | Triple[float] | None = None

pos_noise_range class-attribute instance-attribute 

pos_noise_range: tuple[float, float] | tuple[Triple[float], Triple[float]] | None = None

record_depth class-attribute instance-attribute 

record_depth: bool = False

robot_namespace class-attribute instance-attribute 

robot_namespace: str | None = None

skip_erosion class-attribute instance-attribute 

skip_erosion: bool = False

visibility_constraints class-attribute instance-attribute 

visibility_constraints: dict[str, float] | None = None

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

MlSpacesExpConfig

Bases: Config, ABC

Base configuration class for experiments.

This should be extended to create specific experiment configurations.

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use

CameraConfig class-attribute 

CameraConfig: type = CameraSystemConfig

PolicyConfig class-attribute 

PolicyConfig: type = BasePolicyConfig

RobotConfig class-attribute 

RobotConfig: type = BaseRobotConfig

benchmark_path class-attribute instance-attribute 

benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 

camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 

collision_free_pose_limit: int = 3

config_version class-attribute instance-attribute 

config_version: str = '0.1'

ctrl_dt_ms instance-attribute 

ctrl_dt_ms: float

Default control time step

data_split class-attribute instance-attribute 

data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 

datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 

end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 

environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 

eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 

filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 

fps: float

log_level class-attribute instance-attribute 

log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

no_cached_map class-attribute instance-attribute 

no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs instance-attribute 

num_envs: int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 

num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 

output_dir: Path

Output directory for experiment results

policy_config instance-attribute 

policy_config: BasePolicyConfig

Configuration for policies

policy_dt_ms instance-attribute 

policy_dt_ms: float

Default policy time step

profile class-attribute instance-attribute 

profile: bool = False

Whether to enable profiling

profiler class-attribute instance-attribute 

profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 

robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset instance-attribute 

scene_dataset: str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 

seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms instance-attribute 

sim_dt_ms: float

Default simulation time step

tag abstractmethod property 

tag: str

A string describing the experiment.

task_config instance-attribute 

task_config: AllTaskConfigs

Configuration for tasks

task_config_preset_exp class-attribute instance-attribute 

task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 

task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 

task_horizon: int | None = None

Maximum number of steps per episode (if None, no time limit)

task_sampler_config instance-attribute 

task_sampler_config: BaseMujocoTaskSamplerConfig

Configuration for the task sampler

task_type instance-attribute 

task_type: str

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer instance-attribute 

use_passive_viewer: bool

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 

use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict instance-attribute 

viewer_cam_dict: dict

Dictionary containing viewer camera parameters

wandb_name class-attribute instance-attribute 

wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 

wandb_project: str | None = None

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

freeze_task_config 

freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_config staticmethod 

load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

model_post_init 

model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
170
171
172
173
174
175
176
177
178
179
180
181
182
183
def model_post_init(self, _context: Any) -> None:
    def is_multiple(val_a: float, val_b: float) -> bool:
        if val_b == 0:
            return val_a == 0

        multiplier = round(val_a / val_b)
        return math.isclose(val_a, multiplier * val_b)

    assert is_multiple(self.policy_dt_ms, self.ctrl_dt_ms), (
        "policy_dt_ms must be a multiple of ctrl_dt_ms"
    )
    assert is_multiple(self.ctrl_dt_ms, self.sim_dt_ms), (
        "ctrl_dt_ms must be a multiple of sim_dt"
    )

save_config 

save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickTaskConfig

Bases: BaseMujocoTaskConfig

Configuration for Franka move-to-pose task.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
enable_rendering bool
model_config
object_poses dict[str, list[float]] | None
pickup_obj_goal_pose list[float] | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
place_target_name str | None
receptacle_name str | None
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
succ_pos_threshold float
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool

action_dtype class-attribute instance-attribute 

action_dtype: str = 'float32'

added_objects class-attribute instance-attribute 

added_objects: dict[str, Path] = {}

enable_rendering class-attribute instance-attribute 

enable_rendering: bool = True

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

object_poses class-attribute instance-attribute 

object_poses: dict[str, list[float]] | None = None

pickup_obj_goal_pose class-attribute instance-attribute 

pickup_obj_goal_pose: list[float] | None = None

pickup_obj_name class-attribute instance-attribute 

pickup_obj_name: str | None = None

pickup_obj_start_pose class-attribute instance-attribute 

pickup_obj_start_pose: list[float] | None = None

place_target_name class-attribute instance-attribute 

place_target_name: str | None = None

receptacle_name class-attribute instance-attribute 

receptacle_name: str | None = None

referral_expressions class-attribute instance-attribute 

referral_expressions: dict[str, str] = {}

referral_expressions_priority class-attribute instance-attribute 

referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}

robot_base_pose class-attribute instance-attribute 

robot_base_pose: list[float] | None = None

succ_pos_threshold class-attribute instance-attribute 

succ_pos_threshold: float = 0.01

task_cls class-attribute instance-attribute 

task_cls: type | None = None

tracked_object_names class-attribute instance-attribute 

tracked_object_names: list[str] | None = None

use_sensors class-attribute instance-attribute 

use_sensors: bool = True

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickTaskSamplerConfig

Bases: ObjectCentricTaskSamplerConfig

Configuration for Franka move-to-pose task sampler.

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool

added_pickup_class_max_uids class-attribute instance-attribute 

added_pickup_class_max_uids: int | None = None

added_pickup_class_rank class-attribute instance-attribute 

added_pickup_class_rank: int | list[int] | None = None

added_pickup_namespace class-attribute instance-attribute 

added_pickup_namespace: str = 'pickup/'

added_pickup_objects class-attribute instance-attribute 

added_pickup_objects: list[str] | None = None

base_pose_sampling_radius_range class-attribute instance-attribute 

base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)

check_robot_placement_visibility class-attribute instance-attribute 

check_robot_placement_visibility: bool = True

dataset_name class-attribute instance-attribute 

dataset_name: str = 'procthor-10k'

enable_texture_randomization class-attribute instance-attribute 

enable_texture_randomization: bool = False

episodes_per_added_pickup class-attribute instance-attribute 

episodes_per_added_pickup: int = 1

episodes_per_batch class-attribute instance-attribute 

episodes_per_batch: int = 4

filter_for_grasps class-attribute instance-attribute 

filter_for_grasps: bool = True

grasp_libraries class-attribute instance-attribute 

grasp_libraries: list[str] | None = None

house_inds class-attribute instance-attribute 

house_inds: list[int] | None = list(range(0, 4))

house_variant class-attribute instance-attribute 

house_variant: str = 'ceiling'

license_policy class-attribute instance-attribute 

license_policy: LicensePolicy = LicensePolicy.NONE

max_added_pickup_placement_attempts class-attribute instance-attribute 

max_added_pickup_placement_attempts: int = 100

max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 

max_allowed_sequential_irrecoverable_failures: int = 5

max_allowed_sequential_rollout_failures class-attribute instance-attribute 

max_allowed_sequential_rollout_failures: int = 10

max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 

max_allowed_sequential_task_sampler_failures: int = 10

max_asset_failures class-attribute instance-attribute 

max_asset_failures: int = 10

max_object_placement_attempts class-attribute instance-attribute 

max_object_placement_attempts: int = 200

max_receptacle_attempts class-attribute instance-attribute 

max_receptacle_attempts: int = 10

max_reference_to_added_pickup_dist class-attribute instance-attribute 

max_reference_to_added_pickup_dist: float = 0.5

max_robot_placement_attempts class-attribute instance-attribute 

max_robot_placement_attempts: int = 10

max_robot_to_added_pickup_dist class-attribute instance-attribute 

max_robot_to_added_pickup_dist: float = 0.7

max_robot_to_obj_dist class-attribute instance-attribute 

max_robot_to_obj_dist: float = 0.6

max_robot_to_target_dist class-attribute instance-attribute 

max_robot_to_target_dist: float = 0.6

max_tasks class-attribute instance-attribute 

max_tasks: float = math.inf

max_total_attempts_multiplier class-attribute instance-attribute 

max_total_attempts_multiplier: int = 6

min_object_separation class-attribute instance-attribute 

min_object_separation: float = 0.05

min_reference_to_added_pickup_dist class-attribute instance-attribute 

min_reference_to_added_pickup_dist: float = 0.15

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

num_added_pickups class-attribute instance-attribute 

num_added_pickups: int = 30

objaverse_oversampling_factor class-attribute instance-attribute 

objaverse_oversampling_factor: int = 30

object_placement_radius_range class-attribute instance-attribute 

object_placement_radius_range: tuple[float, float] = (0.1, 0.8)

occupancy_map_cache_size class-attribute instance-attribute 

occupancy_map_cache_size: int = 4

occupancy_map_impl class-attribute instance-attribute 

occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR

pickup_obj_name class-attribute instance-attribute 

pickup_obj_name: str | None = None

pickup_types class-attribute instance-attribute 

pickup_types: list[str] | None = None

place_target_name class-attribute instance-attribute 

place_target_name: str | None = None

placement_volume_name class-attribute instance-attribute 

placement_volume_name: str | None = None

randomize_dynamics class-attribute instance-attribute 

randomize_dynamics: bool = False

randomize_lighting class-attribute instance-attribute 

randomize_lighting: bool = False

randomize_robot_textures class-attribute instance-attribute 

randomize_robot_textures: bool = False

randomize_textures class-attribute instance-attribute 

randomize_textures: bool = False

randomize_textures_all class-attribute instance-attribute 

randomize_textures_all: bool = False

receptacle_name class-attribute instance-attribute 

receptacle_name: str | None = None

receptacle_types class-attribute instance-attribute 

receptacle_types: list[str] = RECEPTACLE_TYPES_THOR

referral_expression_clip_filter class-attribute instance-attribute 

referral_expression_clip_filter: bool = False

robot_object_z_offset class-attribute instance-attribute 

robot_object_z_offset: float = -0.75

robot_object_z_offset_random_max class-attribute instance-attribute 

robot_object_z_offset_random_max: float = 0.25

robot_object_z_offset_random_min class-attribute instance-attribute 

robot_object_z_offset_random_min: float = -0.3

robot_placement_exclusion_threshold class-attribute instance-attribute 

robot_placement_exclusion_threshold: float = 0.15

robot_placement_rotation_range_rad class-attribute instance-attribute 

robot_placement_rotation_range_rad: float = math.radians(45)

robot_safety_radius class-attribute instance-attribute 

robot_safety_radius: float = 0.15

samples_per_house class-attribute instance-attribute 

samples_per_house: int | None = 2

scene_xml_paths class-attribute instance-attribute 

scene_xml_paths: list[str] | None = None

sim_settle_timesteps class-attribute instance-attribute 

sim_settle_timesteps: int = 500

task_batch_size class-attribute instance-attribute 

task_batch_size: int = 1

task_sampler_class class-attribute instance-attribute 

task_sampler_class: type | None = None

verbose class-attribute instance-attribute 

verbose: bool = False

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

model_post_init 

model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RBY1GoProD455CameraSystem

Bases: CameraSystemConfig

Camera system for RBY1 with GoPro head camera and D455 wrist cameras.

Renders at 1024x576 (16:9) to accommodate both: - Head camera: GoPro analogue (4:3, crop to 768x576 in post-processing) - Wrist cameras: D455 analogue (16:9, use full frame)

All cameras include randomization for sim-to-real transfer.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config

cameras class-attribute instance-attribute 

cameras: list[CameraConfig] = [MjcfCameraConfig(name='head_camera', mjcf_name='head_camera', robot_namespace='robot_0/', fov=139.0, fov_noise_degrees=(-3.0, 3.0), pos_noise_range=((-0.01, -0.01, -0.01), (0.01, 0.01, 0.01)), orientation_noise_degrees=(4.0, 4.0, 4.0), skip_erosion=True), MjcfCameraConfig(name='wrist_camera_l', mjcf_name='wrist_camera_l', robot_namespace='robot_0/', fov=58.0, fov_noise_degrees=(-4.0, 4.0), pos_noise_range=((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), orientation_noise_degrees=(8.0, 4.0, 4.0), record_depth=True), MjcfCameraConfig(name='wrist_camera_r', mjcf_name='wrist_camera_r', robot_namespace='robot_0/', fov=58.0, fov_noise_degrees=(-4.0, 4.0), pos_noise_range=((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), orientation_noise_degrees=(8.0, 4.0, 4.0), record_depth=True)]

img_resolution class-attribute instance-attribute 

img_resolution: tuple[int, int] = (1024, 576)

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

add_camera 

add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

get_camera_by_name 

get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RBY1MjcfCameraSystem

Bases: CameraSystemConfig

Camera system using RBY1's built-in MJCF cameras.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config

cameras class-attribute instance-attribute 

cameras: list[CameraConfig] = [MjcfCameraConfig(name='head_camera', mjcf_name='head_camera', robot_namespace='robot_0/', fov=139.0, skip_erosion=True), MjcfCameraConfig(name='wrist_camera_l', mjcf_name='wrist_camera_l', robot_namespace='robot_0/', record_depth=True), MjcfCameraConfig(name='wrist_camera_r', mjcf_name='wrist_camera_r', robot_namespace='robot_0/', record_depth=True), MjcfCameraConfig(name='camera_follower', mjcf_name='camera_follower', robot_namespace='robot_0/')]

img_resolution class-attribute instance-attribute 

img_resolution: tuple[int, int] = (640, 480)

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

add_camera 

add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

get_camera_by_name 

get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RandomizedExocentricCameraConfig

Bases: CameraConfig

Randomized external camera positioned around a workspace center.

Samples camera position within specified ranges around a workspace center. Can use visibility constraints to ensure good views of important objects. CORE ASSUMPTION: workspace center will be sourced from task sampler callback function get_workspace_center you will always be looking at the workspace center (with optional noise).

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
allow_relaxed_constraints bool
azimuth_range tuple[float, float]
distance_range tuple[float, float]
fov float | None
fov_range tuple[float, float] | None
height_range tuple[float, float]
is_warped bool
lookat_noise_range tuple[float, float] | None
max_placement_attempts int
model_config
name str
record_depth bool
skip_erosion bool
visibility_constraints dict[str, float] | None

allow_relaxed_constraints class-attribute instance-attribute 

allow_relaxed_constraints: bool = False

azimuth_range instance-attribute 

azimuth_range: tuple[float, float]

distance_range instance-attribute 

distance_range: tuple[float, float]

fov class-attribute instance-attribute 

fov: float | None = None

fov_range class-attribute instance-attribute 

fov_range: tuple[float, float] | None = None

height_range instance-attribute 

height_range: tuple[float, float]

is_warped class-attribute instance-attribute 

is_warped: bool = False

lookat_noise_range class-attribute instance-attribute 

lookat_noise_range: tuple[float, float] | None = None

max_placement_attempts class-attribute instance-attribute 

max_placement_attempts: int = 100

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

name instance-attribute 

name: str

record_depth class-attribute instance-attribute 

record_depth: bool = False

skip_erosion class-attribute instance-attribute 

skip_erosion: bool = False

visibility_constraints class-attribute instance-attribute 

visibility_constraints: dict[str, float] | None = None

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RobotMountedCameraConfig

Bases: CameraConfig

Camera dynamically mounted to a robot body.

Camera follows the specified reference body with configurable offset and orientation. Can use either lookat-based positioning or quaternion-based orientation.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_offset list[float]
camera_quaternion list[float] | None
fov float | None
is_warped bool
lookat_noise_range tuple[float, float] | None
lookat_offset list[float]
model_config
name str
orientation_noise_degrees float | None
pos_noise_range tuple[float, float] | None
record_depth bool
reference_body_names list[str]
skip_erosion bool
up_axis str
visibility_constraints dict[str, float] | None

camera_offset class-attribute instance-attribute 

camera_offset: list[float] = [0.1, 0.0, -0.15]

camera_quaternion class-attribute instance-attribute 

camera_quaternion: list[float] | None = None

fov class-attribute instance-attribute 

fov: float | None = None

is_warped class-attribute instance-attribute 

is_warped: bool = False

lookat_noise_range class-attribute instance-attribute 

lookat_noise_range: tuple[float, float] | None = None

lookat_offset class-attribute instance-attribute 

lookat_offset: list[float] = [0.0, 0.0, 0.08]

model_config class-attribute instance-attribute 

model_config = ConfigDict(arbitrary_types_allowed=True)

name instance-attribute 

name: str

orientation_noise_degrees class-attribute instance-attribute 

orientation_noise_degrees: float | None = None

pos_noise_range class-attribute instance-attribute 

pos_noise_range: tuple[float, float] | None = None

record_depth class-attribute instance-attribute 

record_depth: bool = False

reference_body_names instance-attribute 

reference_body_names: list[str]

skip_erosion class-attribute instance-attribute 

skip_erosion: bool = False

up_axis class-attribute instance-attribute 

up_axis: str = 'z'

visibility_constraints class-attribute instance-attribute 

visibility_constraints: dict[str, float] | None = None

from_dict classmethod 

from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)

load_from_json classmethod 

load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)

save_to_json 

save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())

to_dict 

to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()

to_json 

to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

abstract_config

Classes:

Name Description
Config

Base configuration class that can be extended for specific configurations.

Config

Bases: BaseModel

Base configuration class that can be extended for specific configurations. Provides methods to convert to dict, json, and to save/load from files.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
model_config
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

abstract_exp_config

Classes:

Name Description
MlSpacesExpConfig

Base configuration class for experiments.

Attributes:

Name Type Description
log

log module-attribute 

log = logging.getLogger(__name__)

MlSpacesExpConfig

Bases: Config, ABC

Base configuration class for experiments.

This should be extended to create specific experiment configurations.

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms instance-attribute 
ctrl_dt_ms: float

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs instance-attribute 
num_envs: int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config instance-attribute 
policy_config: BasePolicyConfig

Configuration for policies

policy_dt_ms instance-attribute 
policy_dt_ms: float

Default policy time step

profile class-attribute instance-attribute 
profile: bool = False

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset instance-attribute 
scene_dataset: str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms instance-attribute 
sim_dt_ms: float

Default simulation time step

tag abstractmethod property 
tag: str

A string describing the experiment.

task_config instance-attribute 
task_config: AllTaskConfigs

Configuration for tasks

task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = None

Maximum number of steps per episode (if None, no time limit)

task_sampler_config instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig

Configuration for the task sampler

task_type instance-attribute 
task_type: str

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer instance-attribute 
use_passive_viewer: bool

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict instance-attribute 
viewer_cam_dict: dict

Dictionary containing viewer camera parameters

wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = None

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
170
171
172
173
174
175
176
177
178
179
180
181
182
183
def model_post_init(self, _context: Any) -> None:
    def is_multiple(val_a: float, val_b: float) -> bool:
        if val_b == 0:
            return val_a == 0

        multiplier = round(val_a / val_b)
        return math.isclose(val_a, multiplier * val_b)

    assert is_multiple(self.policy_dt_ms, self.ctrl_dt_ms), (
        "policy_dt_ms must be a multiple of ctrl_dt_ms"
    )
    assert is_multiple(self.ctrl_dt_ms, self.sim_dt_ms), (
        "ctrl_dt_ms must be a multiple of sim_dt"
    )
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

base_nav_to_obj_config

Example configuration for RBY1 navigation to object data generation using the extracted task sampler. This shows how the scene randomization functionality from the reference script has been properly integrated into the modular task sampler architecture.

Classes:

Name Description
NavToObjBaseConfig

Base configuration for navigation to object data generation tasks.

NavToObjBaseConfig

Bases: MlSpacesExpConfig

Base configuration for navigation to object data generation tasks.

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Camera configuration - using new unified camera system
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_threads int
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
record_videos bool
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset NavToObjTaskConfig | None
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Task sampler configuration (imported from task_sampler_configs.py)
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = RBY1MjcfCameraSystem()

Camera configuration - using new unified camera system

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_threads class-attribute instance-attribute 
num_threads: int = 1
num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = AStarNavToObjPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 200.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

record_videos class-attribute instance-attribute 
record_videos: bool = False
robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = NavToObjTaskConfig(task_cls=NavToObjTask)

Configuration for tasks

task_config_preset class-attribute instance-attribute 
task_config_preset: NavToObjTaskConfig | None = None
task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = NavToObjTaskSamplerConfig(task_sampler_class=NavToObjTaskSampler)

Task sampler configuration (imported from task_sampler_configs.py)

task_type class-attribute instance-attribute 
task_type: str = 'nav_to_obj'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': np.array([0.0, 0.0, 0.5])}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = None

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_nav_to_obj_config.py 
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

base_open_task_configs

Classes:

Name Description
ClosingBaseConfig

Base configuration for closing task data generation.
OpeningBaseConfig

Base configuration for opening task data generation.

ClosingBaseConfig

Bases: PickBaseConfig

Base configuration for closing task data generation.

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset OpeningTaskConfig | None
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = FrankaRandomizedD405D455CameraSystem()

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = OpenClosePlannerPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 66.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = OpeningTaskConfig(task_cls=OpeningTask, task_success_threshold=0.85, joint_index=0, any_inst_of_category=False)

Configuration for tasks

task_config_preset class-attribute instance-attribute 
task_config_preset: OpeningTaskConfig | None = None
task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = OpenTaskSamplerConfig(task_sampler_class=OpenTaskSampler, target_initial_state_open_percentage=0.5)

Configuration for the task sampler

task_type class-attribute instance-attribute 
task_type: str = 'close'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': [0.0, 0.0, 0.5]}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = 'molmo-spaces-data-generation'

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_pick_config.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    # Auto-create profiler instance if profiling is enabled
    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

OpeningBaseConfig

Bases: PickBaseConfig

Base configuration for opening task data generation.

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset OpeningTaskConfig | None
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = FrankaRandomizedD405D455CameraSystem()

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = OpenClosePlannerPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 66.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = OpeningTaskConfig(task_cls=OpeningTask, task_success_threshold=0.15, joint_index=0, any_inst_of_category=True)

Configuration for tasks

task_config_preset class-attribute instance-attribute 
task_config_preset: OpeningTaskConfig | None = None
task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = OpenTaskSamplerConfig(task_sampler_class=OpenTaskSampler, target_initial_state_open_percentage=0)

Configuration for the task sampler

task_type class-attribute instance-attribute 
task_type: str = 'open'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': [0.0, 0.0, 0.5]}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = 'molmo-spaces-data-generation'

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_pick_config.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    # Auto-create profiler instance if profiling is enabled
    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

base_packing_configs

Classes:

Name Description
PackingDataGenConfig

PackingDataGenConfig

Bases: PickBaseConfig

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = FrankaRandomizedD405D455CameraSystem()

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = PickAndPlacePlannerPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 66.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = PackingTaskConfig(task_cls=PackingTask)

Configuration for tasks

task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = PackingTaskSamplerConfig(task_sampler_class=PackingTaskSampler, pickup_types=PICK_AND_PLACE_OBJECTS, samples_per_house=20)

Configuration for the task sampler

task_type class-attribute instance-attribute 
task_type: str = 'packing'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': [0.0, 0.0, 0.5]}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = 'molmo-spaces-data-generation'

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_pick_config.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    # Auto-create profiler instance if profiling is enabled
    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

base_pick_and_place_color_configs

Classes:

Name Description
PickAndPlaceColorDataGenConfig

PickAndPlaceColorDataGenConfig

Bases: PickBaseConfig

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = FrankaRandomizedD405D455CameraSystem()

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir class-attribute instance-attribute 
output_dir: Path = ASSETS_DIR / 'experiment_output' / 'datagen' / 'pick_and_place_color_base_v1'

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = PickAndPlaceColorPlannerPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 66.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = PickAndPlaceColorTaskConfig(task_cls=PickAndPlaceColorTask)

Configuration for tasks

task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = PickAndPlaceColorTaskSamplerConfig(task_sampler_class=PickAndPlaceColorTaskSampler, samples_per_house=20)

Configuration for the task sampler

task_type class-attribute instance-attribute 
task_type: str = 'pick_and_place_color'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': [0.0, 0.0, 0.5]}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = 'molmo-spaces-data-generation'

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_pick_config.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    # Auto-create profiler instance if profiling is enabled
    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

base_pick_and_place_configs

Classes:

Name Description
PickAndPlaceDataGenConfig

PickAndPlaceDataGenConfig

Bases: PickBaseConfig

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = FrankaRandomizedD405D455CameraSystem()

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = PickAndPlacePlannerPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 66.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = PickAndPlaceTaskConfig(task_cls=PickAndPlaceTask)

Configuration for tasks

task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = PickAndPlaceTaskSamplerConfig(task_sampler_class=PickAndPlaceTaskSampler, pickup_types=[], samples_per_house=20)

Configuration for the task sampler

task_type class-attribute instance-attribute 
task_type: str = 'pick_and_place'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': [0.0, 0.0, 0.5]}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = 'molmo-spaces-data-generation'

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_pick_config.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    # Auto-create profiler instance if profiling is enabled
    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

base_pick_and_place_next_to_configs

Classes:

Name Description
PickAndPlaceNextToDataGenConfig

PickAndPlaceNextToDataGenConfig

Bases: PickBaseConfig

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = FrankaRandomizedD405D455CameraSystem()

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = PickAndPlaceNextToPlannerPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 66.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig = FrankaRobotConfig()

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = PickAndPlaceNextToTaskConfig(task_cls=PickAndPlaceNextToTask)

Configuration for tasks

task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = PickAndPlaceNextToTaskSamplerConfig(task_sampler_class=PickAndPlaceNextToTaskSampler, pickup_types=PICK_AND_PLACE_OBJECTS, samples_per_house=20)

Configuration for the task sampler

task_type class-attribute instance-attribute 
task_type: str = 'pick_and_place_next_to'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': [0.0, 0.0, 0.5]}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = 'molmo-spaces-data-generation'

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_pick_config.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    # Auto-create profiler instance if profiling is enabled
    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

base_pick_config

Example configuration for Franka pick-and-place data generation using the extracted task sampler. This shows how the scene randomization functionality from the reference script has been properly integrated into the modular task sampler architecture.

Classes:

Name Description
PickBaseConfig

PickBaseConfig

Bases: MlSpacesExpConfig

Classes:

Name Description
SavedEpisode

Config information describing a single episode

Methods:

Name Description
freeze_task_config
from_dict
load_config
load_from_json
model_post_init
save_config
save_to_json
to_dict
to_json

Attributes:

Name Type Description
CameraConfig type
PolicyConfig type
RobotConfig type
benchmark_path Path | None

Contains a json with a list of fully-specified episodes
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
collision_free_pose_limit int
config_version str
ctrl_dt_ms float

Default control time step
data_split str

Data split to use, e.g. train, val, test
datagen_profiler bool

Whether or not to use the datagen profiler
end_on_success bool

Whether to end episode immediately upon success (overrides task_horizon if True)
environment_light_intensity float

The environment intensity value for the IBL when using the filament-based renderer
eval_runtime_params Any

Evaluation runtime parameters (optional, set during evaluation initialization)
filter_for_successful_trajectories bool

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory
fps float
log_level str

Global logging level: "debug", "info", "warning", "error", "none
model_config
no_cached_map bool

Whether or not to cache the generated thormap to disk after creating it for a datagen run
num_envs int

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)
num_workers int

Number of worker processes for parallel data generation (episode-level parallelism)
output_dir Path

Output directory for experiment results
policy_config BasePolicyConfig

Configuration for policies
policy_dt_ms float

Default policy time step
profile bool

Whether to enable profiling
profiler Profiler | None

Profiler instance (auto-created if profile=True)
robot_config BaseRobotConfig

Configuration for the robot
scene_dataset str

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config
seed int | None

Random seed for task sampling (if None, generates random seed)
sim_dt_ms float

Default simulation time step
tag str

A string describing the experiment.
task_config AllTaskConfigs

Configuration for tasks
task_config_preset_exp AllTaskConfigs | None

Cached config for whole experiment
task_config_preset_scn AllTaskConfigs | None

Cached config for scene
task_horizon int | None

Maximum number of steps per episode (if None, no time limit)
task_sampler_config BaseMujocoTaskSamplerConfig

Configuration for the task sampler
task_type str

Task type: e.g. pick, pick_and_place, etc.
use_passive_viewer bool

Launch passive viewer for rendering
use_wandb bool

Whether or not to use wandb logging
viewer_cam_dict dict

Dictionary containing viewer camera parameters
viewer_camera None
wandb_name str | None

(Optional) The name of the wandb run
wandb_project str | None

(Optional) The name of the wandb project to use
CameraConfig class-attribute 
CameraConfig: type = CameraSystemConfig
PolicyConfig class-attribute 
PolicyConfig: type = BasePolicyConfig
RobotConfig class-attribute 
RobotConfig: type = BaseRobotConfig
benchmark_path class-attribute instance-attribute 
benchmark_path: Path | None = None

Contains a json with a list of fully-specified episodes

camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = FrankaRandomizedD405D455CameraSystem()

Configuration for cameras and sensors

collision_free_pose_limit class-attribute instance-attribute 
collision_free_pose_limit: int = 3
config_version class-attribute instance-attribute 
config_version: str = '0.1'
ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 2.0

Default control time step

data_split class-attribute instance-attribute 
data_split: str = 'train'

Data split to use, e.g. train, val, test

datagen_profiler class-attribute instance-attribute 
datagen_profiler: bool = True

Whether or not to use the datagen profiler

end_on_success class-attribute instance-attribute 
end_on_success: bool = False

Whether to end episode immediately upon success (overrides task_horizon if True)

environment_light_intensity class-attribute instance-attribute 
environment_light_intensity: float = 15000.0

The environment intensity value for the IBL when using the filament-based renderer

eval_runtime_params class-attribute instance-attribute 
eval_runtime_params: Any = None

Evaluation runtime parameters (optional, set during evaluation initialization)

filter_for_successful_trajectories class-attribute instance-attribute 
filter_for_successful_trajectories: bool = True

If True, only save successful trajectories to main output directory (failed episodes may be sampled 1% for debug directory). If False, save all trajectories to main output directory

fps property 
fps: float
log_level class-attribute instance-attribute 
log_level: str = 'info'

Global logging level: "debug", "info", "warning", "error", "none

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
no_cached_map class-attribute instance-attribute 
no_cached_map: bool = False

Whether or not to cache the generated thormap to disk after creating it for a datagen run

num_envs class-attribute instance-attribute 
num_envs: int = 1

Number of batched environments per worker (for vectorized physics in CPUMujocoEnv)

num_workers class-attribute instance-attribute 
num_workers: int = 1

Number of worker processes for parallel data generation (episode-level parallelism)

output_dir instance-attribute 
output_dir: Path

Output directory for experiment results

policy_config class-attribute instance-attribute 
policy_config: BasePolicyConfig = PickPlannerPolicyConfig()

Configuration for policies

policy_dt_ms class-attribute instance-attribute 
policy_dt_ms: float = 66.0

Default policy time step

profile class-attribute instance-attribute 
profile: bool = True

Whether to enable profiling

profiler class-attribute instance-attribute 
profiler: Profiler | None = None

Profiler instance (auto-created if profile=True)

robot_config instance-attribute 
robot_config: BaseRobotConfig

Configuration for the robot

scene_dataset class-attribute instance-attribute 
scene_dataset: str = 'procthor-10k'

Scenes to use, e.g. ithor, procthor-10k, procthor-objaverse. If "user", use the scene_xml_paths in task_sampler_config

seed class-attribute instance-attribute 
seed: int | None = None

Random seed for task sampling (if None, generates random seed)

sim_dt_ms class-attribute instance-attribute 
sim_dt_ms: float = 2.0

Default simulation time step

tag property 
tag: str

A string describing the experiment.

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs = PickTaskConfig(task_cls=PickTask)

Configuration for tasks

task_config_preset_exp class-attribute instance-attribute 
task_config_preset_exp: AllTaskConfigs | None = None

Cached config for whole experiment

task_config_preset_scn class-attribute instance-attribute 
task_config_preset_scn: AllTaskConfigs | None = None

Cached config for scene

task_horizon class-attribute instance-attribute 
task_horizon: int | None = 500

Maximum number of steps per episode (if None, no time limit)

task_sampler_config class-attribute instance-attribute 
task_sampler_config: BaseMujocoTaskSamplerConfig = PickTaskSamplerConfig(task_sampler_class=PickTaskSampler)

Configuration for the task sampler

task_type class-attribute instance-attribute 
task_type: str = 'pick'

Task type: e.g. pick, pick_and_place, etc.

use_passive_viewer class-attribute instance-attribute 
use_passive_viewer: bool = False

Launch passive viewer for rendering

use_wandb class-attribute instance-attribute 
use_wandb: bool = False

Whether or not to use wandb logging

viewer_cam_dict class-attribute instance-attribute 
viewer_cam_dict: dict = {'distance': 5.0, 'azimuth': 45.0, 'elevation': -30.0, 'lookat': [0.0, 0.0, 0.5]}

Dictionary containing viewer camera parameters

viewer_camera class-attribute instance-attribute 
viewer_camera: None = None
wandb_name class-attribute instance-attribute 
wandb_name: str | None = None

(Optional) The name of the wandb run

wandb_project class-attribute instance-attribute 
wandb_project: str | None = 'molmo-spaces-data-generation'

(Optional) The name of the wandb project to use

SavedEpisode

Bases: Config

Config information describing a single episode

The code below this is used for saving episode state so that it can be re-loaded w/o sampling

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_config CameraSystemConfig | None

Configuration for cameras and sensors
model_config
robot_config BaseRobotConfig | None

Configuration for the robot
task_cls_str str | None

(Optional) The name of the task class to be used
task_config AllTaskConfigs | None

Configuration for tasks
camera_config class-attribute instance-attribute 
camera_config: CameraSystemConfig | None = None

Configuration for cameras and sensors

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
robot_config class-attribute instance-attribute 
robot_config: BaseRobotConfig | None = None

Configuration for the robot

task_cls_str class-attribute instance-attribute 
task_cls_str: str | None = None

(Optional) The name of the task class to be used

task_config class-attribute instance-attribute 
task_config: AllTaskConfigs | None = None

Configuration for tasks

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")
freeze_task_config 
freeze_task_config(observation, task: BaseMujocoTask) -> str
Source code in molmo_spaces/configs/abstract_exp_config.py 
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
def freeze_task_config(self, observation, task: BaseMujocoTask) -> str:
    sc = MlSpacesExpConfig.SavedEpisode()

    # NOTE(RMH): deep argument VERY IMPORTANT. Mutates config for future episodes otherwise
    sc.robot_config = self.robot_config.model_copy(deep=True)

    sc.robot_config.robot_cls = None
    sc.robot_config.robot_factory = None
    sc.robot_config.robot_view_factory = None

    sc.robot_config.init_qpos_noise_range = None
    sc.robot_config.init_qpos = observation[0]["qpos"]

    # TODO(wilbert): by now the camera_config shouldn't be None (must be set by the user). It
    # might need to be declared as not optional instead
    assert self.camera_config is not None, "Must have a valid 'self.camera_config' by now"

    sc.camera_config = self.camera_config.model_copy(deep=True)
    for i, camera in enumerate(sc.camera_config.cameras):
        # Some cameras can contain random sampling, e.g. of positions
        # Read the camera's positions and convert them to fixed cameras
        if isinstance(camera, MjcfCameraConfig | RobotMountedCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]

            # TODO(wilbert): uhmm, most of the attributes below are only defined in the
            # RobotMountedCamera cls, not the base Camera cls. For now we're assumming
            # that here we always get a RobotMountedCamera instance, otherwise it would
            # crash anyway bc the parameters expected wouldn't be defined
            assert isinstance(cam, RobotMountedCamera), (
                "Something is wrong here, the 'cam' instance should be of class 'RobotMountedCamera'"
            )

            camera_quat: list[float] = (
                cam.camera_quaternion.tolist()
                if cam.camera_quaternion is not None
                else [1.0, 0.0, 0.0, 0.0]
            )
            new_camera = RobotMountedCameraConfig(
                name=cam.name,
                reference_body_names=list(cam.reference_body_names),
                camera_offset=list(cam.camera_offset),
                lookat_offset=list(cam.lookat_offset),
                camera_quaternion=camera_quat,
                fov=cam.fov,
            )
            sc.camera_config.cameras[i] = new_camera

        elif isinstance(camera, RandomizedExocentricCameraConfig | FixedExocentricCameraConfig):
            cam = task.env.camera_manager.registry[camera.name]
            new_camera = FixedExocentricCameraConfig(
                name=cam.name,
                fov=cam.fov,
                pos=list(cam.pos),
                up=list(cam.up),
                forward=list(cam.forward),
            )
            sc.camera_config.cameras[i] = new_camera
        else:
            raise NotImplementedError(f"Cannot freeze camera of type {type(camera).__name__}")

    obj_poses = {}
    om = task.env.object_managers[task.env.current_batch_index]
    task_objects = om.get_mobile_objects()
    for task_object in task_objects:
        obj_poses[task_object.name] = pose_mat_to_7d(task_object.pose).tolist()
    task.config.task_config.object_poses = obj_poses

    sc.task_config = self.task_config.model_copy(deep=True)
    sc.task_config.task_cls = None
    if self.task_config.task_cls is not None:
        sc.task_cls_str = (
            self.task_config.task_cls.__module__ + "." + self.task_config.task_cls.__name__
        )

    if sc.task_config and sc.task_config.robot_base_pose is None:
        raise RuntimeError("'robot_base_pose' attribute must be in the task_config")

    sc_bytes = pickle.dumps(sc)
    sc_b64 = base64.b64encode(sc_bytes).decode("utf-8")
    return sc_b64
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_config staticmethod 
load_config(output_dir: Path) -> MlSpacesExpConfig
Source code in molmo_spaces/configs/abstract_exp_config.py 
207
208
209
210
211
212
213
214
@staticmethod
def load_config(output_dir: Path) -> MlSpacesExpConfig:
    config_path = output_dir / "experiment_config.pkl"
    with open(config_path, "rb") as f:
        config = pickle.load(f)

    log.info(f"Loaded experiment configuration from {output_dir}")
    return config
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/base_pick_config.py 
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)

    try:
        self.policy_config = self._init_policy_config()
    except RuntimeError as e:
        # Check if this is a CUDA/GPU-related error
        error_msg = str(e)
        if "NVIDIA" in error_msg or "CUDA" in error_msg or "GPU" in error_msg:
            # No GPU available - this is expected on manager nodes that just coordinate jobs
            # Policy config will be initialized later on worker nodes that have GPUs
            print(
                f"Warning: Skipping policy config initialization due to missing GPU: {error_msg}"
            )
            self.policy_config = None  # pyright: ignore[reportAttributeAccessIssue] # ty: ignore
        else:
            raise

    # Auto-create profiler instance if profiling is enabled
    if self.profile and self.profiler is None:
        self.profiler = Profiler()
save_config 
save_config(output_dir: Path | str | None = None) -> None
Source code in molmo_spaces/configs/abstract_exp_config.py 
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def save_config(self, output_dir: Path | str | None = None) -> None:
    if output_dir is None:
        output_dir = self.output_dir

    output_dir = Path(output_dir)
    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
    output_dir.mkdir(parents=True, exist_ok=True)

    # TODO(wilbert): maybe we should save to another format as well. If using pickle, the
    # context of the module is also included, so to import the pkl outside of the project (like
    # to just validate it outside or reuse it) we would get import errors
    config_path = output_dir / f"experiment_config_{timestamp}.pkl"
    with open(config_path, "wb") as f:
        pickle.dump(self, f)

    log.info(f"Saved experiment configuration to {output_dir}")
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

camera_configs

Classes:

Name Description
BimanualYamCameraSystem

Camera system for bimanual YAM robot.
CameraConfig

Base specification for a single camera.
CameraSystemConfig

Complete camera system configuration.
EvalExocentricCameraConfig

Eval exocentric camera whose level-0 pose is derived from a shoulder mount.
EvalRobotMountedCameraConfig

Robot-mounted camera with additional intrinsics perturbation for eval.
FisheyeImpl

Which implementation renders a warped camera's image; see
FisheyeMjcfCameraConfig

MJCF camera carrying the parameters FisheyeImpl.CUBEMAP needs.
FixedExocentricCameraConfig

Fixed external camera at a specific world position.
FrankaDroidCameraSystem

Camera system for Franka with DROID-style fixed cameras.
FrankaEasyRandomizedDroidCameraSystem

Camera system for Franka DROID system with wrist cam (ZED mini) and 2 randomized exo cams (ZED 2/ZED 2i).
FrankaEvalCameraSystem

Unified Franka eval camera system with progressive perturbation (0-100 level).
FrankaGoProD405D455CameraSystem

Camera system for Franka with GoPro and D405 analogue cameras with noise.
FrankaGoProD405RandomizedCameraSystem

Camera system for Franka with D405 wrist cam and 2 randomized GoPro exo cams.
FrankaOmniPurposeCameraSystem

Camera system for Franka DROID system with wrist cam (ZED mini), droid-alike left shoulder cam,
FrankaRandomizedD405D455CameraSystem

Camera system for Franka pick-and-place tasks with wrist cam and 2 randomized exo cams.
FrankaRandomizedDroidCameraSystem

Camera system for Franka DROID system with wrist cam (ZED mini) and 2 randomized exo cams (ZED 2/ZED 2i).
FrankaRobotiq2f85CameraSystem

Camera system for Franka with Robotiq 2f85 wrist cam and 2 randomized GoPro exo cams.
I2rtYamCameraSystem

Camera system for i2rt YAM robot.
MjcfCameraConfig

Camera defined in the MJCF file.
RBY1GoProD455CameraSystem

Camera system for RBY1 with GoPro head camera and D455 wrist cameras.
RBY1MjcfCameraSystem

Camera system using RBY1's built-in MJCF cameras.
RandomizedExocentricCameraConfig

Randomized external camera positioned around a workspace center.
RobotMountedCameraConfig

Camera dynamically mounted to a robot body.

Attributes:

Name Type Description
AllCameraSystems TypeAlias
T
Triple TypeAlias
logger

AllCameraSystems module-attribute 

AllCameraSystems: TypeAlias = RBY1MjcfCameraSystem | RBY1GoProD455CameraSystem | FrankaRandomizedD405D455CameraSystem | FrankaEasyRandomizedDroidCameraSystem | FrankaDroidCameraSystem | FrankaOmniPurposeCameraSystem | FrankaRandomizedDroidCameraSystem | FrankaGoProD405D455CameraSystem | FrankaGoProD405RandomizedCameraSystem | FrankaRobotiq2f85CameraSystem | FrankaEvalCameraSystem | I2rtYamCameraSystem | BimanualYamCameraSystem | FrankaEvalCameraSystem

T module-attribute 

T = TypeVar('T')

Triple module-attribute 

Triple: TypeAlias = tuple[T, T, T]

logger module-attribute 

logger = logging.getLogger(__name__)

BimanualYamCameraSystem

Bases: CameraSystemConfig

Camera system for bimanual YAM robot.

Includes wrist cameras on both arms (defined in yam.xml MJCF) and a robot-mounted exo camera positioned to see both arms and the workspace between them.

Note: Camera offset z must account for the base platform height (0.7m). The exo camera is positioned slightly back and higher to capture both arms.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='left_wrist_camera', mjcf_name='wrist_camera', robot_namespace='robot_0/left_', fov=58.0), MjcfCameraConfig(name='right_wrist_camera', mjcf_name='wrist_camera', robot_namespace='robot_0/right_', fov=58.0), RobotMountedCameraConfig(name='exo_camera', reference_body_names=['robot_0/base', 'robot_0/left_arm'], camera_offset=[0.0, 0.0, 1.56], camera_quaternion=[0.687, 0.1675, -0.1675, -0.687], fov=58.0, visibility_constraints={'__task_objects__': 0.001})]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 368)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

CameraConfig

Bases: Config, ABC

Base specification for a single camera.

Each camera spec defines how one camera should be created and configured. Subclasses implement different camera types (MJCF, robot-mounted, exocentric).

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
fov float | None
is_warped bool
model_config
name str
record_depth bool
skip_erosion bool
visibility_constraints dict[str, float] | None
fov class-attribute instance-attribute 
fov: float | None = None
is_warped class-attribute instance-attribute 
is_warped: bool = False
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
record_depth class-attribute instance-attribute 
record_depth: bool = False
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

CameraSystemConfig

Bases: Config

Complete camera system configuration.

Defines all cameras that should be set up in the environment, along with shared settings like resolution.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = []
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 480)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

EvalExocentricCameraConfig

Bases: FixedExocentricCameraConfig

Eval exocentric camera whose level-0 pose is derived from a shoulder mount.

At runtime the reference body pose is resolved into world-frame pos/forward/up, decomposed into spherical coordinates relative to the workspace center, perturbed according to the spherical noise params, and then placed via the normal fixed-exocentric path.

pos, forward, up default to None here (overriding the required parent fields) because they are computed at runtime.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
azimuth_range tuple[float, float] | None
camera_offset list[float]
camera_quaternion list[float]
distance_range tuple[float, float] | None
forward list[float] | None
fov float | None
fov_range tuple[float, float] | None
height_range tuple[float, float] | None
is_warped bool
lookat_noise_range tuple[float, float] | None
max_placement_attempts int
model_config
name str
orientation_noise_degrees float | Triple[float] | None
pos list[float] | None
pos_noise_range tuple[float, float] | tuple[Triple[float], Triple[float]] | None
record_depth bool
reference_body_names list[str]
skip_erosion bool
up list[float] | None
visibility_constraints dict[str, float] | None
workspace_center_weight float | None
azimuth_range class-attribute instance-attribute 
azimuth_range: tuple[float, float] | None = None
camera_offset class-attribute instance-attribute 
camera_offset: list[float] = [0.1, 0.57, 0.66]
camera_quaternion class-attribute instance-attribute 
camera_quaternion: list[float] = [-0.3633, -0.1241, 0.4263, 0.8191]
distance_range class-attribute instance-attribute 
distance_range: tuple[float, float] | None = None
forward class-attribute instance-attribute 
forward: list[float] | None = None
fov class-attribute instance-attribute 
fov: float | None = None
fov_range class-attribute instance-attribute 
fov_range: tuple[float, float] | None = None
height_range class-attribute instance-attribute 
height_range: tuple[float, float] | None = None
is_warped class-attribute instance-attribute 
is_warped: bool = False
lookat_noise_range class-attribute instance-attribute 
lookat_noise_range: tuple[float, float] | None = None
max_placement_attempts class-attribute instance-attribute 
max_placement_attempts: int = 50
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
orientation_noise_degrees class-attribute instance-attribute 
orientation_noise_degrees: float | Triple[float] | None = None
pos class-attribute instance-attribute 
pos: list[float] | None = None
pos_noise_range class-attribute instance-attribute 
pos_noise_range: tuple[float, float] | tuple[Triple[float], Triple[float]] | None = None
record_depth class-attribute instance-attribute 
record_depth: bool = False
reference_body_names class-attribute instance-attribute 
reference_body_names: list[str] = ['robot_0/fr3_link0']
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
up class-attribute instance-attribute 
up: list[float] | None = None
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
workspace_center_weight class-attribute instance-attribute 
workspace_center_weight: float | None = None
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

EvalRobotMountedCameraConfig

Bases: RobotMountedCameraConfig

Robot-mounted camera with additional intrinsics perturbation for eval.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_offset list[float]
camera_quaternion list[float] | None
fov float | None
fov_noise_degrees tuple[float, float] | None
is_warped bool
lookat_noise_range tuple[float, float] | None
lookat_offset list[float]
model_config
name str
orientation_noise_degrees float | None
pos_noise_range tuple[float, float] | None
record_depth bool
reference_body_names list[str]
skip_erosion bool
up_axis str
visibility_constraints dict[str, float] | None
camera_offset class-attribute instance-attribute 
camera_offset: list[float] = [0.1, 0.0, -0.15]
camera_quaternion class-attribute instance-attribute 
camera_quaternion: list[float] | None = None
fov class-attribute instance-attribute 
fov: float | None = None
fov_noise_degrees class-attribute instance-attribute 
fov_noise_degrees: tuple[float, float] | None = None
is_warped class-attribute instance-attribute 
is_warped: bool = False
lookat_noise_range class-attribute instance-attribute 
lookat_noise_range: tuple[float, float] | None = None
lookat_offset class-attribute instance-attribute 
lookat_offset: list[float] = [0.0, 0.0, 0.08]
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
orientation_noise_degrees class-attribute instance-attribute 
orientation_noise_degrees: float | None = None
pos_noise_range class-attribute instance-attribute 
pos_noise_range: tuple[float, float] | None = None
record_depth class-attribute instance-attribute 
record_depth: bool = False
reference_body_names instance-attribute 
reference_body_names: list[str]
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
up_axis class-attribute instance-attribute 
up_axis: str = 'z'
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FisheyeImpl

Bases: StrEnum

Which implementation renders a warped camera's image; see CameraConfig.fisheye_impl. Ignored unless the camera sets is_warped.

CUBEMAP utils/fisheye_cubemap.py. Composites five wide tile cameras through an OpenCV fisheye model calibrated on the real lens. Needs the five tile cameras present in the MJCF, which is its only real cost. Use this for any new warped camera. The G1 head is the only one today; see FisheyeMjcfCameraConfig. WARPING DEPRECATED -- utils/fisheye_warping.py. Post-distorts one pinhole render with a radial k1..k4 model. Do not use for new cameras.

At matched lens, FOV and render budget CUBEMAP is 5.1x sharper (variance of Laplacian 5476 vs 1070), marginally faster (5.90 vs 6.24 ms/frame), and reaches the G1 head lens's 72.8 deg half-FOV, which WARPING's k1..k4 polynomial caps at 68.26 deg (mlspaces_tests/component_tests/ compare_fisheye_renderers.py). WARPING stays the default only so existing configs keep their recorded behaviour; nothing renders through it today.

Attributes:

Name Type Description
CUBEMAP
WARPING
CUBEMAP class-attribute instance-attribute 
CUBEMAP = 'cubemap'
WARPING class-attribute instance-attribute 
WARPING = 'warping'

FisheyeMjcfCameraConfig

Bases: MjcfCameraConfig

MJCF camera carrying the parameters FisheyeImpl.CUBEMAP needs.

Only worth using with fisheye_impl=CUBEMAP -- the WARPING default reads none of these fields. The G1 head is the one camera wide enough to need it; see G1CameraSystem.

fov stays the pinhole camera's FOV and tile_fov the tile cameras', so both render paths take their FOV from this config and neither drifts.

Methods:

Name Description
cubemap_renderer_kwargs

Every FisheyeRenderer argument except the MuJoCo model, so callers
from_dict
load_from_json
save_to_json
tile_camera_names

Fully-qualified MJCF names of the five tile cameras ("cubemap" backend).
to_dict
to_json

Attributes:

Name Type Description
distortion_noise float | None
fisheye_D list[float]
fisheye_K list[list[float]]
fisheye_image_size tuple[int, int]
fov float | None
fov_noise_degrees tuple[float, float] | None
fov_noise_focal_divisor float
is_warped bool
mjcf_name str
model_config
name str
orientation_noise_degrees float | Triple[float] | None
pos_noise_range tuple[float, float] | tuple[Triple[float], Triple[float]] | None
record_depth bool
robot_namespace str | None
skip_erosion bool
tile_fov float
tile_size int
tile_suffixes tuple[str, str, str, str, str]
visibility_constraints dict[str, float] | None
weight_power float
distortion_noise class-attribute instance-attribute 
distortion_noise: float | None = None
fisheye_D class-attribute instance-attribute 
fisheye_D: list[float] = [-0.02559442829261663, 0.008371943913215045, -0.006921566406199126, 0.0010132813066123071]
fisheye_K class-attribute instance-attribute 
fisheye_K: list[list[float]] = [[801.6382129934864, 0.0, 976.1246839545557], [0.0, 802.1081824931498, 542.7122090223202], [0.0, 0.0, 1.0]]
fisheye_image_size class-attribute instance-attribute 
fisheye_image_size: tuple[int, int] = (1920, 1080)
fov class-attribute instance-attribute 
fov: float | None = None
fov_noise_degrees class-attribute instance-attribute 
fov_noise_degrees: tuple[float, float] | None = None
fov_noise_focal_divisor class-attribute instance-attribute 
fov_noise_focal_divisor: float = 90.0
is_warped class-attribute instance-attribute 
is_warped: bool = False
mjcf_name instance-attribute 
mjcf_name: str
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
orientation_noise_degrees class-attribute instance-attribute 
orientation_noise_degrees: float | Triple[float] | None = None
pos_noise_range class-attribute instance-attribute 
pos_noise_range: tuple[float, float] | tuple[Triple[float], Triple[float]] | None = None
record_depth class-attribute instance-attribute 
record_depth: bool = False
robot_namespace class-attribute instance-attribute 
robot_namespace: str | None = None
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
tile_fov class-attribute instance-attribute 
tile_fov: float = 100.0
tile_size class-attribute instance-attribute 
tile_size: int = 512
tile_suffixes class-attribute instance-attribute 
tile_suffixes: tuple[str, str, str, str, str] = ('center', 'up', 'down', 'left', 'right')
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
weight_power class-attribute instance-attribute 
weight_power: float = 4.0
cubemap_renderer_kwargs 
cubemap_renderer_kwargs(output_h: int, output_w: int) -> dict

Every FisheyeRenderer argument except the MuJoCo model, so callers construct one straight from config rather than restating its parameters.

Output size stays a call argument: it is the resolution the caller wants this frame (CameraSystemConfig.img_resolution), not a property of the lens.

Source code in molmo_spaces/configs/camera_configs.py 
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
def cubemap_renderer_kwargs(self, output_h: int, output_w: int) -> dict:
    """Every FisheyeRenderer argument except the MuJoCo model, so callers
    construct one straight from config rather than restating its parameters.

    Output size stays a call argument: it is the resolution the caller wants
    this frame (CameraSystemConfig.img_resolution), not a property of the
    lens.
    """
    return {
        "tile_cam_names": self.tile_camera_names(),
        "K": self.fisheye_K,
        "D": self.fisheye_D,
        "image_size": self.fisheye_image_size,
        "tile_size": self.tile_size,
        "tile_fovy": self.tile_fov,
        "weight_power": self.weight_power,
        "output_h": output_h,
        "output_w": output_w,
    }
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
tile_camera_names 
tile_camera_names() -> list[str]

Fully-qualified MJCF names of the five tile cameras ("cubemap" backend).

Source code in molmo_spaces/configs/camera_configs.py 
143
144
145
146
def tile_camera_names(self) -> list[str]:
    """Fully-qualified MJCF names of the five tile cameras ("cubemap" backend)."""
    prefix = self.robot_namespace or ""
    return [f"{prefix}{self.mjcf_name}_tile_{s}" for s in self.tile_suffixes]
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FixedExocentricCameraConfig

Bases: CameraConfig

Fixed external camera at a specific world position.

Useful for consistent third-person views, overhead cameras, or monitoring positions. Can optionally add small amounts of noise for data augmentation.

TODO: should this also have a quaternion option? was figuring this would be most useful for fixed eval episodes

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
forward list[float]
fov float | None
is_warped bool
model_config
name str
orientation_noise_degrees float | Triple[float] | None
pos list[float]
pos_noise_range tuple[float, float] | tuple[Triple[float], Triple[float]] | None
record_depth bool
skip_erosion bool
up list[float]
visibility_constraints dict[str, float] | None
forward instance-attribute 
forward: list[float]
fov class-attribute instance-attribute 
fov: float | None = None
is_warped class-attribute instance-attribute 
is_warped: bool = False
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
orientation_noise_degrees class-attribute instance-attribute 
orientation_noise_degrees: float | Triple[float] | None = None
pos instance-attribute 
pos: list[float]
pos_noise_range class-attribute instance-attribute 
pos_noise_range: tuple[float, float] | tuple[Triple[float], Triple[float]] | None = None
record_depth class-attribute instance-attribute 
record_depth: bool = False
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
up instance-attribute 
up: list[float]
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaDroidCameraSystem

Bases: CameraSystemConfig

Camera system for Franka with DROID-style fixed cameras.

Uses wrist camera plus DROID-style exocentric camera mounted to robot base. All cameras are deterministic (no noise) for consistent, reproducible viewpoints. This matches the behavior of the old cameras_fixed_droid=True setting.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='gripper/wrist_camera', robot_namespace='robot_0/', fov=56.74), RobotMountedCameraConfig(name='exo_camera_1', reference_body_names=['robot_0/fr3_link0'], camera_offset=[0.1, 0.57, 0.66], camera_quaternion=[-0.3633, -0.1241, 0.4263, 0.8191], fov=71.0, visibility_constraints={'__task_objects__': 0.001})]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 368)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaEasyRandomizedDroidCameraSystem

Bases: CameraSystemConfig

Camera system for Franka DROID system with wrist cam (ZED mini) and 2 randomized exo cams (ZED 2/ZED 2i).

Uses workspace center from task sampler for dynamic placement. The task sampler should implement get_workspace_center() and resolve_visibility_object() to provide runtime information without modifying the camera config.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='gripper/wrist_camera', robot_namespace='robot_0/', fov=52.0, fov_noise_degrees=(-4.0, 4.0), pos_noise_range=((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), orientation_noise_degrees=(8.0, 4.0, 4.0), record_depth=True), RobotMountedCameraConfig(name='exo_camera_1', reference_body_names=['robot_0/fr3_link0'], camera_offset=[0.1, 0.57, 0.66], camera_quaternion=[-0.3633, -0.1241, 0.4263, 0.8191], fov=71.0, pos_noise_range=(-0.05, 0.05), orientation_noise_degrees=8.0, visibility_constraints={'__task_objects__': 0.001})]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 368)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaEvalCameraSystem

Bases: CameraSystemConfig

Unified Franka eval camera system with progressive perturbation (0-100 level).

Cameras: - 1 wrist camera (FrankaDroid-like) - 1 ZED2-like exocentric cameras (stored pose from episode + perturbation)

The cameras list holds only calibrated (level 0) specs with no randomization ranges. All randomization ranges are in per-reference level, per-camera dicts. Level scaling is applied by apply_eval_camera_randomization_level() in eval_camera_randomization_utils.py using an N-piece linear curve (e.g. 0→low→high→100 for N=3).

Exo camera pos/forward/up are placeholders; the runtime loads stored poses from episode specs.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
ref_level_ranges list[tuple[float, dict[str, dict[str, float | tuple[float, ...] | tuple[tuple[float, ...], ...]]]]]
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='gripper/wrist_camera', robot_namespace='robot_0/', fov=52.0), EvalExocentricCameraConfig(name='exo_camera_1', fov=71.0, visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001})]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 368)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
ref_level_ranges class-attribute 
ref_level_ranges: list[tuple[float, dict[str, dict[str, float | tuple[float, ...] | tuple[tuple[float, ...], ...]]]]] = [(0.0, {'wrist_camera': {'pos_noise_range': ((0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), 'orientation_noise_degrees': (0.0, 0.0, 0.0), 'fov_noise_degrees': (0.0, 0.0)}, 'exo_camera_1': {'azimuth_range': (0.0, 0.0), 'distance_range': (0.0, 0.0), 'height_range': (0.0, 0.0), 'workspace_center_weight': 0.0, 'lookat_noise_range': (0.0, 0.0), 'fov_range': (71.0, 71.0)}}), (10.0, {'wrist_camera': {'pos_noise_range': ((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), 'orientation_noise_degrees': (8.0, 4.0, 4.0), 'fov_noise_degrees': (-4.0, 4.0)}, 'exo_camera_1': {'azimuth_range': (-np.pi / 4, np.pi / 4), 'distance_range': (-0.05, 0.05), 'height_range': (-0.05, 0.05), 'workspace_center_weight': 1.0, 'lookat_noise_range': (-0.01, 0.01), 'fov_range': (71.0, 71.0)}}), (40.0, {'wrist_camera': {'pos_noise_range': ((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), 'orientation_noise_degrees': (8.0, 4.0, 4.0), 'fov_noise_degrees': (-4.0, 4.0)}, 'exo_camera_1': {'azimuth_range': (-np.pi / 2, np.pi / 2), 'distance_range': (-0.5, 0.5), 'height_range': (-0.1, 0.5), 'workspace_center_weight': 1.0, 'lookat_noise_range': (-0.05, 0.05), 'fov_range': (64.0, 72.0)}}), (75.0, {'wrist_camera': {'pos_noise_range': ((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), 'orientation_noise_degrees': (8.0, 4.0, 4.0), 'fov_noise_degrees': (-4.0, 4.0)}, 'exo_camera_1': dict(azimuth_range=(-np.pi, np.pi), distance_range=(-0.5, 1.0), height_range=(-0.2, 0.7), workspace_center_weight=1.0, lookat_noise_range=(-0.1, 0.1), fov_range=(64.0, 72.0))}), (100.0, {'wrist_camera': {'pos_noise_range': ((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), 'orientation_noise_degrees': (8.0, 4.0, 4.0), 'fov_noise_degrees': (-4.0, 4.0)}, 'exo_camera_1': dict(azimuth_range=(-np.pi, np.pi), distance_range=(-0.5, 1.5), height_range=(-0.3, 0.8), workspace_center_weight=1.0, lookat_noise_range=(-0.15, 0.15), fov_range=(64.0, 78.0))})]
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaGoProD405D455CameraSystem

Bases: CameraSystemConfig

Camera system for Franka with GoPro and D405 analogue cameras with noise.

Uses: - D405 analogue wrist camera: VFOV=58°, resolution 640x480, with position and orientation noise - 455 analogue exo camera: VFOV=58°, resolution 640x480, with position and orientation noise but around droid shoulder - GoPro analogue exo camera: VFOV=139°, resolution 640x480, with position and orientation noise

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='gripper/wrist_camera', robot_namespace='robot_0/', fov=58.0, record_depth=True, pos_noise_range=(-0.01, 0.01), orientation_noise_degrees=2.0), RobotMountedCameraConfig(name='exo_camera_1', reference_body_names=['robot_0/fr3_link0'], camera_offset=[0.1, 0.57, 0.66], camera_quaternion=[-0.3633, -0.1241, 0.4263, 0.8191], fov=58.0, is_warped=False, pos_noise_range=(-0.1, 0.1), orientation_noise_degrees=3.0, visibility_constraints={'__task_objects__': 0.001}), RandomizedExocentricCameraConfig(name='exo_camera_2', distance_range=(0.2, 0.5), height_range=(0.1, 0.6), azimuth_range=(0, 2 * np.pi), fov=139.0, is_warped=False, lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, max_placement_attempts=20, allow_relaxed_constraints=False)]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 480)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaGoProD405RandomizedCameraSystem

Bases: CameraSystemConfig

Camera system for Franka with D405 wrist cam and 2 randomized GoPro exo cams.

Uses: - D405 analogue wrist camera: VFOV=58°, resolution 640x480, with position and orientation noise - Two randomized GoPro exo cameras: VFOV=139°, resolution 640x480, with visibility constraints

Workspace center sourced from task sampler, exo cameras positioned to maximize visibility of pickup object and gripper.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='wrist_cam', robot_namespace='robot_0/', fov=58.0, record_depth=True, pos_noise_range=(-0.01, 0.01), orientation_noise_degrees=2.0), RandomizedExocentricCameraConfig(name='exo_camera_1', distance_range=(0.4, 1.0), height_range=(0.4, 0.8), azimuth_range=(0, 2 * np.pi), fov=139.0, is_warped=False, lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, max_placement_attempts=20, allow_relaxed_constraints=False), RandomizedExocentricCameraConfig(name='exo_camera_2', distance_range=(0.4, 1.0), height_range=(0.4, 0.8), azimuth_range=(0, 2 * np.pi), fov=139.0, is_warped=False, lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, max_placement_attempts=20, allow_relaxed_constraints=False)]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 480)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaOmniPurposeCameraSystem

Bases: CameraSystemConfig

Camera system for Franka DROID system with wrist cam (ZED mini), droid-alike left shoulder cam, 2 randomized Zed2 cams, and 1 randomized GoPro cam. Intended such that data with this camera system can be used for a wide variety of purposes and maximally consistent ablations.

Uses workspace center from task sampler for dynamic placement. The task sampler should implement get_workspace_center() and resolve_visibility_object() to provide runtime information without modifying the camera config.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera_zed_mini', mjcf_name='gripper/wrist_camera', robot_namespace='robot_0/', fov=52.0, fov_noise_degrees=(-4.0, 4.0), pos_noise_range=((-0.015, -0.005, -0.02), (0.015, 0.005, 0.02)), orientation_noise_degrees=(8.0, 4.0, 4.0), record_depth=True), RobotMountedCameraConfig(name='droid_shoulder_light_randomization', reference_body_names=['robot_0/fr3_link0'], camera_offset=[0.1, 0.57, 0.66], camera_quaternion=[-0.3633, -0.1241, 0.4263, 0.8191], fov=71.0, pos_noise_range=(-0.05, 0.05), orientation_noise_degrees=8.0, visibility_constraints={'__task_objects__': 0.001}), RandomizedExocentricCameraConfig(name='randomized_zed2_analogue_1', distance_range=(0.2, 0.8), height_range=(0.05, 0.6), azimuth_range=(0, 2 * np.pi), fov_range=(64, 72), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, max_placement_attempts=20, allow_relaxed_constraints=False), RandomizedExocentricCameraConfig(name='randomized_zed2_analogue_2', distance_range=(0.2, 0.8), height_range=(0.05, 0.6), azimuth_range=(0, 2 * np.pi), fov_range=(64, 72), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, max_placement_attempts=20, allow_relaxed_constraints=False), RandomizedExocentricCameraConfig(name='randomized_gopro_analogue_1', distance_range=(0.2, 0.5), height_range=(0.1, 0.6), azimuth_range=(0, 2 * np.pi), fov_range=(137, 140), is_warped=False, lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, max_placement_attempts=20, allow_relaxed_constraints=False)]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 368)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaRandomizedD405D455CameraSystem

Bases: CameraSystemConfig

Camera system for Franka pick-and-place tasks with wrist cam and 2 randomized exo cams.

Uses workspace center from task sampler for dynamic placement. The task sampler should implement get_workspace_center() and resolve_visibility_object() to provide runtime information without modifying the camera config.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='wrist_cam', robot_namespace='robot_0/', fov=58.0, fov_noise_degrees=(-10.0, 10.0), pos_noise_range=(-0.015, 0.015), orientation_noise_degrees=8.0), RandomizedExocentricCameraConfig(name='exo_camera_1', distance_range=(0.2, 0.8), height_range=(0.4, 0.8), azimuth_range=(0, 2 * np.pi), fov_range=(50, 90), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, allow_relaxed_constraints=False), RandomizedExocentricCameraConfig(name='exo_camera_2', distance_range=(0.2, 0.8), height_range=(0.4, 0.8), azimuth_range=(0, 2 * np.pi), fov_range=(50, 90), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, allow_relaxed_constraints=False)]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 368)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaRandomizedDroidCameraSystem

Bases: CameraSystemConfig

Camera system for Franka DROID system with wrist cam (ZED mini) and 2 randomized exo cams (ZED 2/ZED 2i).

Uses workspace center from task sampler for dynamic placement. The task sampler should implement get_workspace_center() and resolve_visibility_object() to provide runtime information without modifying the camera config.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='gripper/wrist_camera', robot_namespace='robot_0/', fov=52.0, fov_noise_degrees=(-4.0, 4.0), pos_noise_range=((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), orientation_noise_degrees=(8.0, 4.0, 4.0)), RandomizedExocentricCameraConfig(name='exo_camera_1', distance_range=(0.2, 0.8), height_range=(0.05, 0.6), azimuth_range=(0, 2 * np.pi), fov_range=(64, 72), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, allow_relaxed_constraints=False), RandomizedExocentricCameraConfig(name='exo_camera_2', distance_range=(0.2, 0.8), height_range=(0.05, 0.6), azimuth_range=(0, 2 * np.pi), fov_range=(64, 72), lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, allow_relaxed_constraints=False), RandomizedExocentricCameraConfig(name='exo_camera_3', distance_range=(0.2, 0.5), height_range=(0.1, 0.6), azimuth_range=(0, 2 * np.pi), fov_range=(137, 140), is_warped=False, lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__task_objects__': 0.0001, '__gripper__': 0.0001}, max_placement_attempts=20, allow_relaxed_constraints=False)]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 368)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaRobotiq2f85CameraSystem

Bases: CameraSystemConfig

Camera system for Franka with Robotiq 2f85 wrist cam and 2 randomized GoPro exo cams.

Uses: - Robotiq 2f85 wrist camera: VFOV=56.74°, resolution 1280x720, with position and orientation noise - Two randomized GoPro exo cameras: VFOV=139°, resolution 640x480, with visibility constraints

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='wrist_camera', robot_namespace='robot_0/'), RandomizedExocentricCameraConfig(name='exo_camera_1', distance_range=(0.4, 1.0), height_range=(0.4, 0.8), azimuth_range=(0, 2 * np.pi), fov=139.0, is_warped=False, lookat_noise_range=(-0.1, 0.1), visibility_constraints={'__pickup_object__': 0.001}, max_placement_attempts=200, allow_relaxed_constraints=True)]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 480)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

I2rtYamCameraSystem

Bases: CameraSystemConfig

Camera system for i2rt YAM robot.

Uses robot-mounted exo camera since YAM doesn't have built-in MJCF cameras. The exo camera is mounted relative to the robot base (mocap body at ground level).

Note: Camera offset z must account for the base platform height (0.7m). To achieve similar viewing angle as Franka DROID (camera at ~1.24m total height), we use z offset = 0.7 (platform) + 0.5 (above platform) = 1.2m

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='wrist_camera', mjcf_name='wrist_camera', robot_namespace='robot_0/'), RobotMountedCameraConfig(name='exo_camera_1', reference_body_names=['robot_0/base', 'robot_0/arm'], camera_offset=[0.1, 0.5, 1.2], camera_quaternion=[-0.3633, -0.1241, 0.4263, 0.8191], fov=71.0, visibility_constraints={'__task_objects__': 0.001})]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 480)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

MjcfCameraConfig

Bases: CameraConfig

Camera defined in the MJCF file.

This references a camera that already exists in the scene MJCF or robot MJCF. Useful for cameras with fixed mounting in robot models.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
fov float | None
fov_noise_degrees tuple[float, float] | None
is_warped bool
mjcf_name str
model_config
name str
orientation_noise_degrees float | Triple[float] | None
pos_noise_range tuple[float, float] | tuple[Triple[float], Triple[float]] | None
record_depth bool
robot_namespace str | None
skip_erosion bool
visibility_constraints dict[str, float] | None
fov class-attribute instance-attribute 
fov: float | None = None
fov_noise_degrees class-attribute instance-attribute 
fov_noise_degrees: tuple[float, float] | None = None
is_warped class-attribute instance-attribute 
is_warped: bool = False
mjcf_name instance-attribute 
mjcf_name: str
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
orientation_noise_degrees class-attribute instance-attribute 
orientation_noise_degrees: float | Triple[float] | None = None
pos_noise_range class-attribute instance-attribute 
pos_noise_range: tuple[float, float] | tuple[Triple[float], Triple[float]] | None = None
record_depth class-attribute instance-attribute 
record_depth: bool = False
robot_namespace class-attribute instance-attribute 
robot_namespace: str | None = None
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RBY1GoProD455CameraSystem

Bases: CameraSystemConfig

Camera system for RBY1 with GoPro head camera and D455 wrist cameras.

Renders at 1024x576 (16:9) to accommodate both: - Head camera: GoPro analogue (4:3, crop to 768x576 in post-processing) - Wrist cameras: D455 analogue (16:9, use full frame)

All cameras include randomization for sim-to-real transfer.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='head_camera', mjcf_name='head_camera', robot_namespace='robot_0/', fov=139.0, fov_noise_degrees=(-3.0, 3.0), pos_noise_range=((-0.01, -0.01, -0.01), (0.01, 0.01, 0.01)), orientation_noise_degrees=(4.0, 4.0, 4.0), skip_erosion=True), MjcfCameraConfig(name='wrist_camera_l', mjcf_name='wrist_camera_l', robot_namespace='robot_0/', fov=58.0, fov_noise_degrees=(-4.0, 4.0), pos_noise_range=((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), orientation_noise_degrees=(8.0, 4.0, 4.0), record_depth=True), MjcfCameraConfig(name='wrist_camera_r', mjcf_name='wrist_camera_r', robot_namespace='robot_0/', fov=58.0, fov_noise_degrees=(-4.0, 4.0), pos_noise_range=((-0.015, -0.005, -0.01), (0.015, 0.005, 0.01)), orientation_noise_degrees=(8.0, 4.0, 4.0), record_depth=True)]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (1024, 576)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RBY1MjcfCameraSystem

Bases: CameraSystemConfig

Camera system using RBY1's built-in MJCF cameras.

Methods:

Name Description
add_camera

Add a camera specification to the system.
from_dict
get_camera_by_name

Get a camera spec by name.
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
cameras list[CameraConfig]
img_resolution tuple[int, int]
model_config
cameras class-attribute instance-attribute 
cameras: list[CameraConfig] = [MjcfCameraConfig(name='head_camera', mjcf_name='head_camera', robot_namespace='robot_0/', fov=139.0, skip_erosion=True), MjcfCameraConfig(name='wrist_camera_l', mjcf_name='wrist_camera_l', robot_namespace='robot_0/', record_depth=True), MjcfCameraConfig(name='wrist_camera_r', mjcf_name='wrist_camera_r', robot_namespace='robot_0/', record_depth=True), MjcfCameraConfig(name='camera_follower', mjcf_name='camera_follower', robot_namespace='robot_0/')]
img_resolution class-attribute instance-attribute 
img_resolution: tuple[int, int] = (640, 480)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
add_camera 
add_camera(camera_spec: CameraConfig) -> None

Add a camera specification to the system.

Source code in molmo_spaces/configs/camera_configs.py 
299
300
301
def add_camera(self, camera_spec: CameraConfig) -> None:
    """Add a camera specification to the system."""
    self.cameras.append(camera_spec)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_camera_by_name 
get_camera_by_name(name: str) -> CameraConfig | None

Get a camera spec by name.

Source code in molmo_spaces/configs/camera_configs.py 
303
304
305
306
307
308
def get_camera_by_name(self, name: str) -> CameraConfig | None:
    """Get a camera spec by name."""
    for camera in self.cameras:
        if camera.name == name:
            return camera
    return None
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RandomizedExocentricCameraConfig

Bases: CameraConfig

Randomized external camera positioned around a workspace center.

Samples camera position within specified ranges around a workspace center. Can use visibility constraints to ensure good views of important objects. CORE ASSUMPTION: workspace center will be sourced from task sampler callback function get_workspace_center you will always be looking at the workspace center (with optional noise).

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
allow_relaxed_constraints bool
azimuth_range tuple[float, float]
distance_range tuple[float, float]
fov float | None
fov_range tuple[float, float] | None
height_range tuple[float, float]
is_warped bool
lookat_noise_range tuple[float, float] | None
max_placement_attempts int
model_config
name str
record_depth bool
skip_erosion bool
visibility_constraints dict[str, float] | None
allow_relaxed_constraints class-attribute instance-attribute 
allow_relaxed_constraints: bool = False
azimuth_range instance-attribute 
azimuth_range: tuple[float, float]
distance_range instance-attribute 
distance_range: tuple[float, float]
fov class-attribute instance-attribute 
fov: float | None = None
fov_range class-attribute instance-attribute 
fov_range: tuple[float, float] | None = None
height_range instance-attribute 
height_range: tuple[float, float]
is_warped class-attribute instance-attribute 
is_warped: bool = False
lookat_noise_range class-attribute instance-attribute 
lookat_noise_range: tuple[float, float] | None = None
max_placement_attempts class-attribute instance-attribute 
max_placement_attempts: int = 100
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
record_depth class-attribute instance-attribute 
record_depth: bool = False
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RobotMountedCameraConfig

Bases: CameraConfig

Camera dynamically mounted to a robot body.

Camera follows the specified reference body with configurable offset and orientation. Can use either lookat-based positioning or quaternion-based orientation.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
camera_offset list[float]
camera_quaternion list[float] | None
fov float | None
is_warped bool
lookat_noise_range tuple[float, float] | None
lookat_offset list[float]
model_config
name str
orientation_noise_degrees float | None
pos_noise_range tuple[float, float] | None
record_depth bool
reference_body_names list[str]
skip_erosion bool
up_axis str
visibility_constraints dict[str, float] | None
camera_offset class-attribute instance-attribute 
camera_offset: list[float] = [0.1, 0.0, -0.15]
camera_quaternion class-attribute instance-attribute 
camera_quaternion: list[float] | None = None
fov class-attribute instance-attribute 
fov: float | None = None
is_warped class-attribute instance-attribute 
is_warped: bool = False
lookat_noise_range class-attribute instance-attribute 
lookat_noise_range: tuple[float, float] | None = None
lookat_offset class-attribute instance-attribute 
lookat_offset: list[float] = [0.0, 0.0, 0.08]
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str
orientation_noise_degrees class-attribute instance-attribute 
orientation_noise_degrees: float | None = None
pos_noise_range class-attribute instance-attribute 
pos_noise_range: tuple[float, float] | None = None
record_depth class-attribute instance-attribute 
record_depth: bool = False
reference_body_names instance-attribute 
reference_body_names: list[str]
skip_erosion class-attribute instance-attribute 
skip_erosion: bool = False
up_axis class-attribute instance-attribute 
up_axis: str = 'z'
visibility_constraints class-attribute instance-attribute 
visibility_constraints: dict[str, float] | None = None
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

dummy_config

Classes:

Name Description
DummyPolicyConfig

Policy config that uses DummyPolicy for testing.

DummyPolicyConfig

Bases: BasePolicyConfig

Policy config that uses DummyPolicy for testing.

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
force_enable_depth bool

Whether or not to require all cameras to record depth
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'dummy'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/dummy_config.py 
14
15
16
17
18
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.policy_cls is None:
        self.policy_cls = DummyPolicy
        self.policy_factory = make_lenient(DummyPolicy)  # pyright: ignore[reportIncompatibleMethodOverride]
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

policy_configs

Policy configuration classes for MolmoSpaces experiments.

Classes:

Name Description
AStarNavToObjPolicyConfig

Configuration for A* navigation policy (discrete grid-based planner).
BasePolicyConfig

Base configuration for policies.
BrownianMotionPolicyConfig

Policy that applies Gaussian noise increments over noop control, resulting in Brownian motion.
CuroboOpenClosePlannerPolicyConfig
CuroboPickAndPlacePlannerPolicyConfig
DoorOpeningPolicyConfig

Configuration for RBY1 door opening planner policy.
DummyPolicyConfig

Policy config that uses DummyPolicy for testing.
NavToObjPlannerPolicyConfig

Base configuration for navigation to object planner policies.
ObjectManipulationPlannerPolicyConfig

Configuration for Franka pick planner policy.
OpenClosePlannerPolicyConfig
PickAndPlaceColorPlannerPolicyConfig
PickAndPlaceNextToPlannerPolicyConfig
PickAndPlacePlannerPolicyConfig
PickPlannerPolicyConfig

AStarNavToObjPolicyConfig

Bases: NavToObjPlannerPolicyConfig

Configuration for A* navigation policy (discrete grid-based planner).

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
downscale int
force_enable_depth bool

Whether or not to require all cameras to record depth
map_path str | None
model_config
num_recovery_steps int
path_interpolation_density int
path_max_inter_waypoint_angle float
path_max_inter_waypoint_dist float
path_min_dist_to_target_center float
plan_fail_after_waypoint_steps int
plan_fail_max_dist_delta float
plan_max_retries int
plan_stick_to_original_target bool
planner_config AStarPlannerConfig
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
recovery_motion_backward_distance float
verbose bool
downscale class-attribute instance-attribute 
downscale: int = 5
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

map_path class-attribute instance-attribute 
map_path: str | None = None
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_recovery_steps class-attribute instance-attribute 
num_recovery_steps: int = 8
path_interpolation_density class-attribute instance-attribute 
path_interpolation_density: int = 1
path_max_inter_waypoint_angle class-attribute instance-attribute 
path_max_inter_waypoint_angle: float = float(np.deg2rad(10))
path_max_inter_waypoint_dist class-attribute instance-attribute 
path_max_inter_waypoint_dist: float = 0.25
path_min_dist_to_target_center class-attribute instance-attribute 
path_min_dist_to_target_center: float = 0.8
plan_fail_after_waypoint_steps class-attribute instance-attribute 
plan_fail_after_waypoint_steps: int = 10
plan_fail_max_dist_delta class-attribute instance-attribute 
plan_fail_max_dist_delta: float = 0.01
plan_max_retries class-attribute instance-attribute 
plan_max_retries: int = 3
plan_stick_to_original_target class-attribute instance-attribute 
plan_stick_to_original_target: bool = False
planner_config class-attribute instance-attribute 
planner_config: AStarPlannerConfig = AStarPlannerConfig()
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

recovery_motion_backward_distance class-attribute instance-attribute 
recovery_motion_backward_distance: float = 0.02
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
418
419
420
421
422
423
424
425
426
427
def model_post_init(self, __context) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.solvers.navigation.astar_planner_policy import (
            AStarSmoothPlannerPolicy,
        )

        self.policy_cls = AStarSmoothPlannerPolicy
        self.policy_factory = AStarSmoothPlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

BasePolicyConfig

Bases: Config

Base configuration for policies.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
force_enable_depth bool

Whether or not to require all cameras to record depth
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls instance-attribute 
policy_cls: type[BasePolicy] | None
policy_factory instance-attribute 
policy_factory: PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type instance-attribute 
policy_type: str

Type of the policy, e.g., "planner", "teleop", "learned", etc.

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

BrownianMotionPolicyConfig

Bases: BasePolicyConfig

Policy that applies Gaussian noise increments over noop control, resulting in Brownian motion.

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
force_enable_depth bool

Whether or not to require all cameras to record depth
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
std float
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'dummy'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

std class-attribute instance-attribute 
std: float = 0.1
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None
Source code in molmo_spaces/configs/policy_configs.py 
454
455
456
457
458
459
460
def model_post_init(self, _context: Any) -> None:
    super().model_post_init(_context)
    if self.policy_cls is None:
        from molmo_spaces.policy.dummy_policy import BrownianMotionPolicy

        self.policy_cls = BrownianMotionPolicy
        self.policy_factory = make_lenient(BrownianMotionPolicy)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

CuroboOpenClosePlannerPolicyConfig

Bases: OpenClosePlannerPolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
attach_obj bool
batch_size int
debug_poses bool
enable_collision_avoidance bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_horizontal_cost_weight float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_closed_pos float
gripper_closed_tolerance float
gripper_empty_threshold float
gripper_open_duration float
left_curobo_planner_config CuroboPlannerConfig | None
left_planner_joint_ranges dict[str, tuple]
max_batch_plan_attempts int
max_grasping_timesteps int
max_height_adjustment_steps int
max_opening_timesteps int
max_planning_reattempts int
max_retries int
max_sequential_ik_failures int
max_settle_steps int
max_steps_per_waypoint int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
right_curobo_planner_config CuroboPlannerConfig | None
right_planner_joint_ranges dict[str, tuple]
server_timeout float | None
server_urls list[str]
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
velocity_constraints dict[str, float]
verbose bool
attach_obj class-attribute instance-attribute 
attach_obj: bool = False
batch_size class-attribute instance-attribute 
batch_size: int = 4
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
enable_collision_avoidance class-attribute instance-attribute 
enable_collision_avoidance: bool = True
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 0.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_horizontal_cost_weight class-attribute instance-attribute 
grasp_horizontal_cost_weight: float = 10.0
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = ['droid']
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.05
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 2.0
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_closed_pos class-attribute instance-attribute 
gripper_closed_pos: float = 0.0
gripper_closed_tolerance class-attribute instance-attribute 
gripper_closed_tolerance: float = 0.005
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
left_curobo_planner_config class-attribute instance-attribute 
left_curobo_planner_config: CuroboPlannerConfig | None = None
left_planner_joint_ranges class-attribute instance-attribute 
left_planner_joint_ranges: dict[str, tuple] = {'base': (0, 3), 'left_arm': (3, 10)}
max_batch_plan_attempts class-attribute instance-attribute 
max_batch_plan_attempts: int = 4
max_grasping_timesteps class-attribute instance-attribute 
max_grasping_timesteps: int = 5
max_height_adjustment_steps class-attribute instance-attribute 
max_height_adjustment_steps: int = 10
max_opening_timesteps class-attribute instance-attribute 
max_opening_timesteps: int = 5
max_planning_reattempts class-attribute instance-attribute 
max_planning_reattempts: int = 2
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
max_settle_steps class-attribute instance-attribute 
max_settle_steps: int = 5
max_steps_per_waypoint class-attribute instance-attribute 
max_steps_per_waypoint: int = 10
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.2
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.05
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.02
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
right_curobo_planner_config class-attribute instance-attribute 
right_curobo_planner_config: CuroboPlannerConfig | None = None
right_planner_joint_ranges class-attribute instance-attribute 
right_planner_joint_ranges: dict[str, tuple] = {'base': (0, 3), 'right_arm': (3, 10)}
server_timeout class-attribute instance-attribute 
server_timeout: float | None = 300.0
server_urls class-attribute instance-attribute 
server_urls: list[str] = ['jupiter-cs-aus-107.reviz.ai2.in:10002']
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.08
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.04
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
velocity_constraints class-attribute instance-attribute 
velocity_constraints: dict[str, float] = {'base': 0.5, 'head': 0.5, 'right_arm': 0.5, 'left_arm': 0.5}
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
130
131
132
133
134
135
136
137
138
139
def model_post_init(self, __context) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.solvers.object_manipulation.open_close_planner_policy import (
            OpenClosePlannerPolicy,
        )

        self.policy_cls = OpenClosePlannerPolicy
        self.policy_factory = OpenClosePlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

CuroboPickAndPlacePlannerPolicyConfig

Bases: PickAndPlacePlannerPolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
attach_obj bool
batch_size int
debug_poses bool
enable_collision_avoidance bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_closed_pos float
gripper_closed_tolerance float
gripper_empty_threshold float
gripper_open_duration float
left_curobo_planner_config CuroboPlannerConfig | None
left_planner_joint_ranges dict[str, tuple]
max_batch_plan_attempts int
max_grasping_timesteps int
max_opening_timesteps int
max_planning_reattempts int
max_retries int
max_sequential_ik_failures int
max_settle_steps int
max_steps_per_waypoint int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
right_curobo_planner_config CuroboPlannerConfig | None
right_planner_joint_ranges dict[str, tuple]
server_timeout float | None
server_urls list[str]
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
velocity_constraints dict[str, float]
verbose bool
attach_obj class-attribute instance-attribute 
attach_obj: bool = False
batch_size class-attribute instance-attribute 
batch_size: int = 4
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
enable_collision_avoidance class-attribute instance-attribute 
enable_collision_avoidance: bool = True
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 8.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.01
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 0.5
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_closed_pos class-attribute instance-attribute 
gripper_closed_pos: float = 0.0
gripper_closed_tolerance class-attribute instance-attribute 
gripper_closed_tolerance: float = 0.005
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
left_curobo_planner_config class-attribute instance-attribute 
left_curobo_planner_config: CuroboPlannerConfig | None = None
left_planner_joint_ranges class-attribute instance-attribute 
left_planner_joint_ranges: dict[str, tuple] = {'base': (0, 3), 'left_arm': (3, 10)}
max_batch_plan_attempts class-attribute instance-attribute 
max_batch_plan_attempts: int = 4
max_grasping_timesteps class-attribute instance-attribute 
max_grasping_timesteps: int = 5
max_opening_timesteps class-attribute instance-attribute 
max_opening_timesteps: int = 5
max_planning_reattempts class-attribute instance-attribute 
max_planning_reattempts: int = 5
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
max_settle_steps class-attribute instance-attribute 
max_settle_steps: int = 5
max_steps_per_waypoint class-attribute instance-attribute 
max_steps_per_waypoint: int = 10
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.5
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.05
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.02
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
right_curobo_planner_config class-attribute instance-attribute 
right_curobo_planner_config: CuroboPlannerConfig | None = None
right_planner_joint_ranges class-attribute instance-attribute 
right_planner_joint_ranges: dict[str, tuple] = {'base': (0, 3), 'right_arm': (3, 10)}
server_timeout class-attribute instance-attribute 
server_timeout: float | None = 300.0
server_urls class-attribute instance-attribute 
server_urls: list[str] = ['jupiter-cs-aus-107.reviz.ai2.in:10002']
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.2
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.08
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
velocity_constraints class-attribute instance-attribute 
velocity_constraints: dict[str, float] = {'base': 0.5, 'head': 0.5, 'right_arm': 0.5, 'left_arm': 0.5}
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
162
163
164
165
166
167
168
169
170
171
def model_post_init(self, __context) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.solvers.object_manipulation.pick_and_place_planner_policy import (
            PickAndPlacePlannerPolicy,
        )

        self.policy_cls = PickAndPlacePlannerPolicy
        self.policy_factory = PickAndPlacePlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

DoorOpeningPolicyConfig

Bases: BasePolicyConfig

Configuration for RBY1 door opening planner policy.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
articulation_deltas list[float]
enable_collision_avoidance bool
first_pushing_articulation_deltas list[float]
force_enable_depth bool

Whether or not to require all cameras to record depth
gripper_closed_pos float
gripper_closed_tolerance float
joint_position_tolerance float
left_curobo_planner_config CuroboPlannerConfig | None
left_gripper_close_command dict
left_gripper_open_command dict
left_planner_joint_ranges dict[str, tuple]
max_grasping_timesteps int
max_planning_failures int
max_steps_per_waypoint int
model_config
num_recovery_steps int
plan_in_robot_frame bool
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
pre_grasp_distance float
recovery_motion_backward_distance float
relevant_collision_objects_radius float
right_curobo_planner_config CuroboPlannerConfig | None
right_gripper_close_command dict
right_gripper_open_command dict
right_planner_joint_ranges dict[str, tuple]
velocity_constraints dict[str, float]
verbose bool
articulation_deltas class-attribute instance-attribute 
articulation_deltas: list[float] = [np.pi / 180.0 * 13.0]
enable_collision_avoidance class-attribute instance-attribute 
enable_collision_avoidance: bool = True
first_pushing_articulation_deltas class-attribute instance-attribute 
first_pushing_articulation_deltas: list[float] = [np.pi / 180.0 * 30.0]
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

gripper_closed_pos class-attribute instance-attribute 
gripper_closed_pos: float = 0.0
gripper_closed_tolerance class-attribute instance-attribute 
gripper_closed_tolerance: float = 0.005
joint_position_tolerance class-attribute instance-attribute 
joint_position_tolerance: float = 0.0275
left_curobo_planner_config class-attribute instance-attribute 
left_curobo_planner_config: CuroboPlannerConfig | None = None
left_gripper_close_command class-attribute instance-attribute 
left_gripper_close_command: dict = {'left_gripper': 100.0}
left_gripper_open_command class-attribute instance-attribute 
left_gripper_open_command: dict = {'left_gripper': -100.0}
left_planner_joint_ranges class-attribute instance-attribute 
left_planner_joint_ranges: dict[str, tuple] = {'base': (0, 3), 'left_arm': (3, 10)}
max_grasping_timesteps class-attribute instance-attribute 
max_grasping_timesteps: int = 5
max_planning_failures class-attribute instance-attribute 
max_planning_failures: int = 15
max_steps_per_waypoint class-attribute instance-attribute 
max_steps_per_waypoint: int = 10
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_recovery_steps class-attribute instance-attribute 
num_recovery_steps: int = 8
plan_in_robot_frame class-attribute instance-attribute 
plan_in_robot_frame: bool = True
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

pre_grasp_distance class-attribute instance-attribute 
pre_grasp_distance: float = -0.18
recovery_motion_backward_distance class-attribute instance-attribute 
recovery_motion_backward_distance: float = 0.02
relevant_collision_objects_radius class-attribute instance-attribute 
relevant_collision_objects_radius: float = 3.0
right_curobo_planner_config class-attribute instance-attribute 
right_curobo_planner_config: CuroboPlannerConfig | None = None
right_gripper_close_command class-attribute instance-attribute 
right_gripper_close_command: dict = {'right_gripper': 100.0}
right_gripper_open_command class-attribute instance-attribute 
right_gripper_open_command: dict = {'right_gripper': -100.0}
right_planner_joint_ranges class-attribute instance-attribute 
right_planner_joint_ranges: dict[str, tuple] = {'base': (0, 3), 'right_arm': (3, 10)}
velocity_constraints class-attribute instance-attribute 
velocity_constraints: dict[str, float] = {'base': 0.5, 'head': 0.5, 'right_arm': 0.5, 'left_arm': 0.5}
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

DummyPolicyConfig

Bases: BasePolicyConfig

Policy config that uses DummyPolicy for testing.

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
force_enable_depth bool

Whether or not to require all cameras to record depth
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'dummy'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/policy_configs.py 
437
438
439
440
441
442
443
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.dummy_policy import DummyPolicy

        self.policy_cls = DummyPolicy
        self.policy_factory = make_lenient(DummyPolicy)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

NavToObjPlannerPolicyConfig

Bases: BasePolicyConfig

Base configuration for navigation to object planner policies.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
force_enable_depth bool

Whether or not to require all cameras to record depth
model_config
num_recovery_steps int
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
recovery_motion_backward_distance float
verbose bool
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_recovery_steps class-attribute instance-attribute 
num_recovery_steps: int = 8
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

recovery_motion_backward_distance class-attribute instance-attribute 
recovery_motion_backward_distance: float = 0.02
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

ObjectManipulationPlannerPolicyConfig

Bases: BasePolicyConfig

Configuration for Franka pick planner policy.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
debug_poses bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_empty_threshold float
gripper_open_duration float
max_retries int
max_sequential_ik_failures int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
verbose bool
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 8.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.01
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 2.0
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.1
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.05
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.04
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.2
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.08
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

OpenClosePlannerPolicyConfig

Bases: ObjectManipulationPlannerPolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
debug_poses bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_horizontal_cost_weight float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_empty_threshold float
gripper_open_duration float
max_retries int
max_sequential_ik_failures int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
verbose bool
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 0.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_horizontal_cost_weight class-attribute instance-attribute 
grasp_horizontal_cost_weight: float = 10.0
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = ['droid']
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.05
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 0.0
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.2
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.05
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.04
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.08
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.04
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
130
131
132
133
134
135
136
137
138
139
def model_post_init(self, __context) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.solvers.object_manipulation.open_close_planner_policy import (
            OpenClosePlannerPolicy,
        )

        self.policy_cls = OpenClosePlannerPolicy
        self.policy_factory = OpenClosePlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceColorPlannerPolicyConfig

Bases: PickAndPlacePlannerPolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
debug_poses bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_empty_threshold float
gripper_open_duration float
max_retries int
max_sequential_ik_failures int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
verbose bool
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 8.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.01
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 2.0
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.5
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.05
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.04
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.2
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.08
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
281
282
283
284
285
286
287
288
def model_post_init(self, _context: Any) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    from molmo_spaces.policy.solvers.object_manipulation.pick_and_place_color_planner_policy import (
        PickAndPlaceColorPlannerPolicy,
    )

    self.policy_cls = PickAndPlaceColorPlannerPolicy
    self.policy_factory = PickAndPlaceColorPlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceNextToPlannerPolicyConfig

Bases: PickAndPlacePlannerPolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
debug_poses bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_empty_threshold float
gripper_open_duration float
max_retries int
max_sequential_ik_failures int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
verbose bool
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 8.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.01
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 2.0
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.5
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.05
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.04
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.2
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.08
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
268
269
270
271
272
273
274
275
def model_post_init(self, _context: Any) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    from molmo_spaces.policy.solvers.object_manipulation.pick_and_place_next_to_planner_policy import (
        PickAndPlaceNextToPlannerPolicy,
    )

    self.policy_cls = PickAndPlaceNextToPlannerPolicy
    self.policy_factory = PickAndPlaceNextToPlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlacePlannerPolicyConfig

Bases: ObjectManipulationPlannerPolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
debug_poses bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_empty_threshold float
gripper_open_duration float
max_retries int
max_sequential_ik_failures int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
verbose bool
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 8.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.01
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 2.0
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.5
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.05
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.04
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.2
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.08
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
162
163
164
165
166
167
168
169
170
171
def model_post_init(self, __context) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.solvers.object_manipulation.pick_and_place_planner_policy import (
            PickAndPlacePlannerPolicy,
        )

        self.policy_cls = PickAndPlacePlannerPolicy
        self.policy_factory = PickAndPlacePlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickPlannerPolicyConfig

Bases: ObjectManipulationPlannerPolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
debug_poses bool
end_z_offset float
filter_colliding_grasps bool
filter_feasible_grasps bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasp_base_pos list[float]
grasp_collision_batch_size int
grasp_collision_max_grasps int
grasp_com_dist_cost_weight float
grasp_feasibility_batch_size int
grasp_feasibility_max_grasps int
grasp_height float
grasp_length float
grasp_libraries list[str] | None
grasp_pos_cost_weight float
grasp_rot_cost_weight float
grasp_vertical_cost_weight float
grasp_width float
grasp_xy_noise float
grasp_yaw_noise float
grasp_z_offset float
gripper_close_duration float
gripper_empty_threshold float
gripper_open_duration float
max_retries int
max_sequential_ik_failures int
model_config
move_settle_time float
phase_timeout float
place_z_offset float
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
postgrasp_height_noise float
postgrasp_z_offset float
pregrasp_height_noise float
pregrasp_z_offset float
randomize_grasp bool
speed_fast float
speed_slow float
tcp_pos_err_threshold float
tcp_rot_err_threshold float
verbose bool
debug_poses class-attribute instance-attribute 
debug_poses: bool = False
end_z_offset class-attribute instance-attribute 
end_z_offset: float = 0.05
filter_colliding_grasps class-attribute instance-attribute 
filter_colliding_grasps: bool = True
filter_feasible_grasps class-attribute instance-attribute 
filter_feasible_grasps: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasp_base_pos class-attribute instance-attribute 
grasp_base_pos: list[float] = [0.0, 0.0, -0.04]
grasp_collision_batch_size class-attribute instance-attribute 
grasp_collision_batch_size: int = 128
grasp_collision_max_grasps class-attribute instance-attribute 
grasp_collision_max_grasps: int = 512
grasp_com_dist_cost_weight class-attribute instance-attribute 
grasp_com_dist_cost_weight: float = 8.0
grasp_feasibility_batch_size class-attribute instance-attribute 
grasp_feasibility_batch_size: int = 256
grasp_feasibility_max_grasps class-attribute instance-attribute 
grasp_feasibility_max_grasps: int = 256
grasp_height class-attribute instance-attribute 
grasp_height: float = 0.01
grasp_length class-attribute instance-attribute 
grasp_length: float = 0.05
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
grasp_pos_cost_weight class-attribute instance-attribute 
grasp_pos_cost_weight: float = 1.0
grasp_rot_cost_weight class-attribute instance-attribute 
grasp_rot_cost_weight: float = 0.01
grasp_vertical_cost_weight class-attribute instance-attribute 
grasp_vertical_cost_weight: float = 2.0
grasp_width class-attribute instance-attribute 
grasp_width: float = 0.08
grasp_xy_noise class-attribute instance-attribute 
grasp_xy_noise: float = 0.02
grasp_yaw_noise class-attribute instance-attribute 
grasp_yaw_noise: float = 0.5
grasp_z_offset class-attribute instance-attribute 
grasp_z_offset: float = 0.03
gripper_close_duration class-attribute instance-attribute 
gripper_close_duration: float = 0.5
gripper_empty_threshold class-attribute instance-attribute 
gripper_empty_threshold: float = 0.002
gripper_open_duration class-attribute instance-attribute 
gripper_open_duration: float = 0.25
max_retries class-attribute instance-attribute 
max_retries: int = 3
max_sequential_ik_failures class-attribute instance-attribute 
max_sequential_ik_failures: int = 8
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
move_settle_time class-attribute instance-attribute 
move_settle_time: float = 0.1
phase_timeout class-attribute instance-attribute 
phase_timeout: float = 10.0
place_z_offset class-attribute instance-attribute 
place_z_offset: float = 0.07
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'planner'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

postgrasp_height_noise class-attribute instance-attribute 
postgrasp_height_noise: float = 0.02
postgrasp_z_offset class-attribute instance-attribute 
postgrasp_z_offset: float = 0.08
pregrasp_height_noise class-attribute instance-attribute 
pregrasp_height_noise: float = 0.03
pregrasp_z_offset class-attribute instance-attribute 
pregrasp_z_offset: float = 0.04
randomize_grasp class-attribute instance-attribute 
randomize_grasp: bool = False
speed_fast class-attribute instance-attribute 
speed_fast: float = 0.2
speed_slow class-attribute instance-attribute 
speed_slow: float = 0.08
tcp_pos_err_threshold class-attribute instance-attribute 
tcp_pos_err_threshold: float = 0.1
tcp_rot_err_threshold class-attribute instance-attribute 
tcp_rot_err_threshold: float = np.radians(30.0)
verbose class-attribute instance-attribute 
verbose: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs.py 
146
147
148
149
150
151
152
153
154
155
def model_post_init(self, __context) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.solvers.object_manipulation.pick_planner_policy import (
            PickPlannerPolicy,
        )

        self.policy_cls = PickPlannerPolicy
        self.policy_factory = PickPlannerPolicy
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

policy_configs_baselines

Classes:

Name Description
BimanualYamPiPolicyConfig

Configuration for BimanualYamPiPolicy using LeRobot gRPC server.
CAPPolicyConfig
DreamZeroPolicyConfig
PiPolicyConfig
TeleopPolicyConfig

BimanualYamPiPolicyConfig

Bases: BasePolicyConfig

Configuration for BimanualYamPiPolicy using LeRobot gRPC server.

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
buffer_length int
camera_mapping dict
checkpoint_path str
force_enable_depth bool

Whether or not to require all cameras to record depth
grasping_type str
model_config
name str
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
remote_config dict
buffer_length class-attribute instance-attribute 
buffer_length: int = 50
camera_mapping class-attribute instance-attribute 
camera_mapping: dict = dict(left_wrist_camera='observation.images.left', right_wrist_camera='observation.images.right', exo_camera='observation.images.top')
checkpoint_path class-attribute instance-attribute 
checkpoint_path: str = 'Jiafei1224/ppack200k'
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasping_type class-attribute instance-attribute 
grasping_type: str = 'binary'
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str = 'bimanual_yam_pi'
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'learned'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

remote_config class-attribute instance-attribute 
remote_config: dict = dict(host='triton-cs-aus-454.reviz.ai2.in', port=8060, policy_type='pi05', device='cuda')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs_baselines.py 
131
132
133
134
135
136
137
138
139
140
def model_post_init(self, _context: Any) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(_context)
    if self.policy_cls is None:
        from molmo_spaces.policy.learned_policy.bimanual_yam_pi_policy import (
            BimanualYamPiPolicy,
        )

        self.policy_cls = BimanualYamPiPolicy
        self.policy_factory = make_lenient(BimanualYamPiPolicy)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

CAPPolicyConfig

Bases: BasePolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
exo_vlm bool
force_enable_depth bool

Whether or not to require all cameras to record depth
grasping_threshold float
grasping_type str
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
remote_config dict
use_vlm bool
exo_vlm class-attribute instance-attribute 
exo_vlm: bool = True
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasping_threshold class-attribute instance-attribute 
grasping_threshold: float = 0.7
grasping_type class-attribute instance-attribute 
grasping_type: str = 'binary'
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'learned'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

remote_config class-attribute instance-attribute 
remote_config: dict = dict(host='localhost', port=8765)
use_vlm class-attribute instance-attribute 
use_vlm: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs_baselines.py 
62
63
64
65
66
67
68
69
def model_post_init(self, __context) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(__context)
    if self.policy_cls is None:
        from molmo_spaces.policy.learned_policy.cap_policy import CAP_Policy

        self.policy_cls = CAP_Policy
        self.policy_factory = make_lenient(CAP_Policy)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

DreamZeroPolicyConfig

Bases: BasePolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
checkpoint_path str
chunk_size int
force_enable_depth bool

Whether or not to require all cameras to record depth
grasping_threshold float
grasping_type str
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
remote_config dict
checkpoint_path class-attribute instance-attribute 
checkpoint_path: str = 'checkpoints/dreamzero'
chunk_size class-attribute instance-attribute 
chunk_size: int = 24
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasping_threshold class-attribute instance-attribute 
grasping_threshold: float = 0.5
grasping_type class-attribute instance-attribute 
grasping_type: str = 'binary'
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'learned'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

remote_config class-attribute instance-attribute 
remote_config: dict = dict(host='localhost', port=0)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs_baselines.py 
42
43
44
45
46
47
48
49
def model_post_init(self, _context: Any) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(_context)
    if self.policy_cls is None:
        from molmo_spaces.policy.learned_policy.dreamzero_policy import DreamZero_Policy

        self.policy_cls = DreamZero_Policy
        self.policy_factory = make_lenient(DreamZero_Policy)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PiPolicyConfig

Bases: BasePolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
checkpoint_path str
chunk_size int
force_enable_depth bool

Whether or not to require all cameras to record depth
grasping_threshold float
grasping_type str
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
remote_config dict | None
checkpoint_path class-attribute instance-attribute 
checkpoint_path: str = 'checkpoints/pi'
chunk_size class-attribute instance-attribute 
chunk_size: int = 8
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

grasping_threshold class-attribute instance-attribute 
grasping_threshold: float = 0.5
grasping_type class-attribute instance-attribute 
grasping_type: str = 'binary'
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'learned'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

remote_config class-attribute instance-attribute 
remote_config: dict | None = dict(host='localhost', port=8080)
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs_baselines.py 
21
22
23
24
25
26
27
28
def model_post_init(self, _context: Any) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(_context)
    if self.policy_cls is None:
        from molmo_spaces.policy.learned_policy.pi_policy import PI_Policy

        self.policy_cls = PI_Policy
        self.policy_factory = make_lenient(PI_Policy)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

TeleopPolicyConfig

Bases: BasePolicyConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init

Set policy_cls after initialization to avoid circular imports.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
device str
force_enable_depth bool

Whether or not to require all cameras to record depth
model_config
policy_cls type[BasePolicy] | None
policy_factory PolicyFactory | None

Factory function to create the policy instance from a config and task, can be same as policy_cls
policy_type str

Type of the policy, e.g., "planner", "teleop", "learned", etc.
pos_sensitivity float
product_id int
rot_sensitivity float
rot_step float
step_size float
device class-attribute instance-attribute 
device: str = 'keyboard'
force_enable_depth class-attribute instance-attribute 
force_enable_depth: bool = False

Whether or not to require all cameras to record depth

model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
policy_cls class-attribute instance-attribute 
policy_cls: type[BasePolicy] | None = None
policy_factory class-attribute instance-attribute 
policy_factory: PolicyFactory | None = None

Factory function to create the policy instance from a config and task, can be same as policy_cls

policy_type class-attribute instance-attribute 
policy_type: str = 'teleop'

Type of the policy, e.g., "planner", "teleop", "learned", etc.

pos_sensitivity class-attribute instance-attribute 
pos_sensitivity: float = 0.005
product_id class-attribute instance-attribute 
product_id: int = 50741
rot_sensitivity class-attribute instance-attribute 
rot_sensitivity: float = 0.02
rot_step class-attribute instance-attribute 
rot_step: float = 0.02
step_size class-attribute instance-attribute 
step_size: float = 0.005
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any) -> None

Set policy_cls after initialization to avoid circular imports.

Source code in molmo_spaces/configs/policy_configs_baselines.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def model_post_init(self, _context: Any) -> None:
    """Set policy_cls after initialization to avoid circular imports."""
    super().model_post_init(_context)
    if self.policy_cls is None:
        if self.device == "keyboard":
            from molmo_spaces.policy.learned_policy.keyboard_policy import Keyboard_Policy

            self.policy_cls = Keyboard_Policy
            self.policy_factory = make_lenient(Keyboard_Policy)
        elif self.device == "spacemouse":
            from molmo_spaces.policy.learned_policy.spacemouse_policy import SpaceMouse_Policy

            self.policy_cls = SpaceMouse_Policy
            self.policy_factory = make_lenient(SpaceMouse_Policy)
        elif self.device == "phone":
            from molmo_spaces.policy.learned_policy.phone_policy import Phone_Policy

            self.policy_cls = Phone_Policy
            self.policy_factory = make_lenient(Phone_Policy)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

robot_configs

Robot configuration classes for MolmoSpaces experiments.

This module contains: - ActionNoiseConfig: TCP-bounded noise configuration for arm actions - BaseRobotConfig: Base configuration for all robots - Robot-specific configs: FrankaRobotConfig, RBY1Config, FloatingRUMRobotConfig

Classes:

Name Description
ActionNoiseConfig

Configuration for action noise injection.
BaseRobotConfig

Base configuration for robot setup.
BimanualYamRobotConfig

Configuration for bimanual YAM robot (two 6-DOF arms with parallel grippers).
FloatingRUMRobotConfig
FloatingRobotiq2f85RobotConfig
FrankaCAPRobotConfig

Configuration for Franka FR3 robot.
FrankaRobotConfig

Configuration for Franka FR3 robot.
I2rtYamRobotConfig

Configuration for i2rt YAM 6-DOF robot.
MobileFrankaRobotConfig
RBY1Config

Configuration for RBY1 robot.
RBY1MConfig

Configuration for RBY1M i.e. mecanum wheel robot.
RBY1MOpenCloseConfig

RBY1M config for open/close tasks.

ActionNoiseConfig

Bases: Config

Configuration for action noise injection.

This noise model supports: - Arm noise: TCP-bounded noise that maps through Jacobian to joint space - Base noise: Planar noise applied directly to (x, y, theta) commands

Noise is proportional to the commanded action magnitude

noise_std = action_scale_factor * ||delta||

When the commanded delta is zero, no noise is applied.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_scale_factor float
base_action_scale_factor float
enabled bool
max_base_position_noise float
max_base_rotation_noise float
max_tcp_position_noise float
max_tcp_rotation_noise float
model_config
rotation_noise_scale float
action_scale_factor class-attribute instance-attribute 
action_scale_factor: float = 0.1
base_action_scale_factor class-attribute instance-attribute 
base_action_scale_factor: float = 0.1
enabled class-attribute instance-attribute 
enabled: bool = True
max_base_position_noise class-attribute instance-attribute 
max_base_position_noise: float = 0.02
max_base_rotation_noise class-attribute instance-attribute 
max_base_rotation_noise: float = 0.05
max_tcp_position_noise class-attribute instance-attribute 
max_tcp_position_noise: float = 0.02
max_tcp_rotation_noise class-attribute instance-attribute 
max_tcp_rotation_noise: float = 0.1
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
rotation_noise_scale class-attribute instance-attribute 
rotation_noise_scale: float = 0.1
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

BaseRobotConfig

Bases: Config

Base configuration for robot setup.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
command_mode instance-attribute 
command_mode: dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = False
init_qpos instance-attribute 
init_qpos: dict[str, list[float]]
init_qpos_noise_range instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name instance-attribute 
name: str | None
robot_cls instance-attribute 
robot_cls: type[Robot] | None
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None
robot_namespace instance-attribute 
robot_namespace: str

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory instance-attribute 
robot_view_factory: RobotViewFactory | None
robot_xml_path instance-attribute 
robot_xml_path: Path
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
117
118
119
120
def model_post_init(self, _context):
    """Ensure action_noise_config is always initialized, even when loading from old configs."""
    if self.action_noise_config is None:
        object.__setattr__(self, "action_noise_config", ActionNoiseConfig())
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

BimanualYamRobotConfig

Bases: BaseRobotConfig

Configuration for bimanual YAM robot (two 6-DOF arms with parallel grippers).

The bimanual YAM consists of two YAM arms positioned 44cm apart, both facing forward.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
base_size list[float] | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
base_size class-attribute instance-attribute 
base_size: list[float] | None = [0.3, 0.8, 0.7]
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'arm': 'joint_position', 'gripper': 'joint_position'}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'left_arm': [0.0624, 0.0109, 0.1707, -0.5938, 0.411, 0.3401], 'right_arm': [0.0006, 0.0147, 0.1669, -0.6407, 0.0746, 0.1516], 'left_gripper': [0.03914, 0.0], 'right_gripper': [0.04068, 0.0]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = None
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'i2rt_yam'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = BimanualYamRobot
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = BimanualYamRobot
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = BimanualYamRobotView
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('bimanual_yam.xml')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
436
437
438
439
440
441
def model_post_init(self, _context: Any):
    super().model_post_init(_context)
    if "gripper" in self.command_mode:
        assert self.command_mode["gripper"] == "joint_position"
    if "arm" in self.command_mode:
        assert self.command_mode["arm"] in ["joint_position", "joint_rel_position"]
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FloatingRUMRobotConfig

Bases: BaseRobotConfig

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
command_mode dict

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
ctrl_dt_ms float
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
command_mode class-attribute instance-attribute 
command_mode: dict = {}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 50.0
force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = False
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list] = {'gripper': [0.0, 0.0]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'floating_rum'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = FloatingRUMRobot
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = FloatingRUMRobot
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = FloatingRUMRobotView
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('model.xml')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
117
118
119
120
def model_post_init(self, _context):
    """Ensure action_noise_config is always initialized, even when loading from old configs."""
    if self.action_noise_config is None:
        object.__setattr__(self, "action_noise_config", ActionNoiseConfig())
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FloatingRobotiq2f85RobotConfig

Bases: BaseRobotConfig

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
action_spec dict[str, int]
command_mode dict

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
ctrl_dt_ms float
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, BaseRobotConfig], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
action_spec class-attribute instance-attribute 
action_spec: dict[str, int] = {'base': 7, 'gripper': 2}
command_mode class-attribute instance-attribute 
command_mode: dict = {}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

ctrl_dt_ms class-attribute instance-attribute 
ctrl_dt_ms: float = 50.0
force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = False
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list] = {'gripper': [0.00296, 0.00296]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'floating_robotiq'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = FloatingRobotiqRobot
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, BaseRobotConfig], Robot] | None = FloatingRobotiqRobot
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = FloatingRobotiq2f85RobotView
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('model.xml')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
117
118
119
120
def model_post_init(self, _context):
    """Ensure action_noise_config is always initialized, even when loading from old configs."""
    if self.action_noise_config is None:
        object.__setattr__(self, "action_noise_config", ActionNoiseConfig())
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaCAPRobotConfig

Bases: BaseRobotConfig

Configuration for Franka FR3 robot.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
base_size list[float] | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
base_size class-attribute instance-attribute 
base_size: list[float] | None = [0.5, 0.5, 0.58]
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'arm': 'joint_position', 'gripper': 'joint_position'}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'arm': [0, -1.5, 0.116, -2.45, 0, 0.842, 0.965], 'gripper': [0.00296, 0.00296]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {'arm': [0.025, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175]}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'franka_cap'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = FrankaRobot
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = FrankaRobot
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = FrankaCAPRobotView
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('model.xml')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
245
246
247
248
249
250
def model_post_init(self, _context: Any):
    super().model_post_init(_context)
    if "gripper" in self.command_mode:
        assert self.command_mode["gripper"] == "joint_position"
    if "arm" in self.command_mode:
        assert self.command_mode["arm"] in ["joint_position", "joint_rel_position"]
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

FrankaRobotConfig

Bases: BaseRobotConfig

Configuration for Franka FR3 robot.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
base_size list[float] | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
perturb_texture_probability float
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
base_size class-attribute instance-attribute 
base_size: list[float] | None = [0.5, 0.5, 0.58]
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'arm': 'joint_position', 'gripper': 'joint_position'}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'arm': [0, -0.7853, 0, -2.35619, 0, 1.57079, 0.0], 'gripper': [0.00296, 0.00296]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {'arm': [0.025, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175]}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'franka_droid'
perturb_texture_probability class-attribute instance-attribute 
perturb_texture_probability: float = 0.7
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = FrankaRobot
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = FrankaRobot
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = FrankaDroidRobotView
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('model.xml')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
169
170
171
172
173
174
def model_post_init(self, _context: Any):
    super().model_post_init(_context)
    if "gripper" in self.command_mode:
        assert self.command_mode["gripper"] == "joint_position"
    if "arm" in self.command_mode:
        assert self.command_mode["arm"] in ["joint_position", "joint_rel_position"]
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

I2rtYamRobotConfig

Bases: BaseRobotConfig

Configuration for i2rt YAM 6-DOF robot.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
base_size list[float] | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
base_size class-attribute instance-attribute 
base_size: list[float] | None = [0.3, 0.3, 0.7]
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'arm': 'joint_position', 'gripper': 'joint_position'}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'arm': [0.0, 1.047, 1.047, 0.1, -0.1, 0.0], 'gripper': [0.0, 0.0]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = None
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'i2rt_yam'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = I2rtYamRobot
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = I2rtYamRobot
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = I2rtYamRobotView
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('yam.xml')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context: Any)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
397
398
399
400
401
402
def model_post_init(self, _context: Any):
    super().model_post_init(_context)
    if "gripper" in self.command_mode:
        assert self.command_mode["gripper"] == "joint_position"
    if "arm" in self.command_mode:
        assert self.command_mode["arm"] in ["joint_position", "joint_rel_position"]
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

MobileFrankaRobotConfig

Bases: BaseRobotConfig

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
base_control_params dict[str, dict[str, float]]
base_size list[float]
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
base_control_params class-attribute instance-attribute 
base_control_params: dict[str, dict[str, float]] = {'base_x_act': {'kp': 25000, 'damping_ratio': 1.0, 'ctrlrange': 25}, 'base_y_act': {'kp': 25000, 'damping_ratio': 1.0, 'ctrlrange': 25}, 'base_theta_act': {'kp': 5000, 'damping_ratio': 1.0}}
base_size class-attribute instance-attribute 
base_size: list[float] = [0.5, 0.5, 0.58]
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'base': 'holo_joint_planar_position', 'arm': 'joint_position', 'gripper': 'joint_position'}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'base': [0, 0, 0], 'arm': [0, -0.7853, 0, -2.35619, 0, 1.57079, 0.0], 'gripper': [0.00296, 0.00296]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {'arm': [0.025, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175]}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'franka_droid'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = MobileFrankaRobot
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = MobileFrankaRobot
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = MobileFrankaDroidRobotView
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('model.xml')
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
117
118
119
120
def model_post_init(self, _context):
    """Ensure action_noise_config is always initialized, even when loading from old configs."""
    if self.action_noise_config is None:
        object.__setattr__(self, "action_noise_config", ActionNoiseConfig())
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RBY1Config

Bases: BaseRobotConfig

Configuration for RBY1 robot.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
use_holo_base bool
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'arm': 'joint_position', 'gripper': 'joint_position', 'base': 'holo_joint_planar_position', 'head': None}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'base': [0.0, 0.0, 0.0], 'head': [0.0, 0.6], 'left_arm': [0.5, 0.0, 0.0, -2.3, 0.0, -0.5, 0.0], 'left_gripper': [-0.05], 'right_arm': [0.5, 0.0, 0.0, -2.3, 0.0, -0.5, 0.0], 'right_gripper': [-0.05], 'torso': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {'base': [0.0, 0.0, 0.0], 'head': [0.2, 0.2], 'left_arm': [0.05, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175], 'left_gripper': [0.01], 'right_arm': [0.05, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175], 'right_gripper': [0.01], 'torso': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'rby1'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = RBY1
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = RBY1
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = None
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('rby1_site_control.xml')
use_holo_base class-attribute instance-attribute 
use_holo_base: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
311
312
313
def model_post_init(self, _context):
    super().model_post_init(_context)
    self.robot_view_factory = partial(RBY1RobotView, holo_base=self.use_holo_base)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RBY1MConfig

Bases: RBY1Config

Configuration for RBY1M i.e. mecanum wheel robot.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
use_holo_base bool
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'arm': 'joint_position', 'gripper': 'joint_position', 'base': 'holo_joint_planar_position', 'head': None}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'base': [0.0, 0.0, 0.0], 'head': [0.0, 0.6], 'left_arm': [0.5, 0.0, 0.0, -2.3, 0.0, -0.5, 0.0], 'left_gripper': [-0.05], 'right_arm': [0.5, 0.0, 0.0, -2.3, 0.0, -0.5, 0.0], 'right_gripper': [-0.05], 'torso': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {'base': [0.0, 0.0, 0.0], 'head': [0.2, 0.2], 'left_arm': [0.05, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175], 'left_gripper': [0.01], 'right_arm': [0.05, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175], 'right_gripper': [0.01], 'torso': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'rby1m'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = RBY1
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = RBY1
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = None
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('rby1_v1.2_site_control.xml')
use_holo_base class-attribute instance-attribute 
use_holo_base: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
311
312
313
def model_post_init(self, _context):
    super().model_post_init(_context)
    self.robot_view_factory = partial(RBY1RobotView, holo_base=self.use_holo_base)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RBY1MOpenCloseConfig

Bases: RBY1MConfig

RBY1M config for open/close tasks.

Uses single-scalar torso height control (torso_1 = torso_3 = h, torso_2 = -2*h) instead of commanding all 6 torso joints independently.

Methods:

Name Description
from_dict
get_robot_dir

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
get_robot_xml_path

Get the full path to the robot XML file.
load_from_json
model_post_init

Ensure action_noise_config is always initialized, even when loading from old configs.
save_to_json
to_dict
to_json

Attributes:

Name Type Description
K_damping list[float] | None
K_stiffness list[float] | None
action_noise_config ActionNoiseConfig | None
command_mode dict[str, str | None]

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'
force_limit list[float] | None
gravcomp bool
init_qpos dict[str, list[float]]
init_qpos_noise_range dict[str, list[float]] | None
model_config
name str | None
robot_cls type[Robot] | None
robot_dir Path | None
robot_factory Callable[[MjData, Any], Robot] | None
robot_namespace str

Namespace used to differentiate between one or multiple robots and the environment
robot_view_factory RobotViewFactory | None
robot_xml_path Path
use_holo_base bool
K_damping class-attribute instance-attribute 
K_damping: list[float] | None = None
K_stiffness class-attribute instance-attribute 
K_stiffness: list[float] | None = None
action_noise_config class-attribute instance-attribute 
action_noise_config: ActionNoiseConfig | None = None
command_mode class-attribute instance-attribute 
command_mode: dict[str, str | None] = {'arm': 'joint_rel_position', 'gripper': 'joint_position', 'base': 'holo_joint_rel_planar_position', 'head': None, 'torso': 'height'}

move_group to command_mode e.g., 'joint', 'cartesian', 'velocity'

force_limit class-attribute instance-attribute 
force_limit: list[float] | None = None
gravcomp class-attribute instance-attribute 
gravcomp: bool = True
init_qpos class-attribute instance-attribute 
init_qpos: dict[str, list[float]] = {'base': [0.0, 0.0, 0.0], 'head': [0.0, 0.6], 'left_arm': [0.5, 0.0, 0.0, -2.3, 0.0, -0.5, 0.0], 'left_gripper': [-0.05], 'right_arm': [0.5, 0.0, 0.0, -2.3, 0.0, -0.5, 0.0], 'right_gripper': [-0.05], 'torso': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}
init_qpos_noise_range class-attribute instance-attribute 
init_qpos_noise_range: dict[str, list[float]] | None = {'base': [0.0, 0.0, 0.0], 'head': [0.2, 0.2], 'left_arm': [0.05, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175], 'left_gripper': [0.01], 'right_arm': [0.05, 0.05, 0.075, 0.1, 0.125, 0.15, 0.175], 'right_gripper': [0.01], 'torso': [0.0, 0.0, 0.0, 0.0, 0.0, 0.0]}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
name class-attribute instance-attribute 
name: str | None = 'rby1m'
robot_cls class-attribute instance-attribute 
robot_cls: type[Robot] | None = RBY1
robot_dir class-attribute instance-attribute 
robot_dir: Path | None = None
robot_factory class-attribute instance-attribute 
robot_factory: Callable[[MjData, Any], Robot] | None = RBY1
robot_namespace class-attribute instance-attribute 
robot_namespace: str = 'robot_0/'

Namespace used to differentiate between one or multiple robots and the environment

robot_view_factory class-attribute instance-attribute 
robot_view_factory: RobotViewFactory | None = None
robot_xml_path class-attribute instance-attribute 
robot_xml_path: Path = Path('rby1_v1.2_site_control.xml')
use_holo_base class-attribute instance-attribute 
use_holo_base: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
get_robot_dir 
get_robot_dir() -> Path

Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.

Source code in molmo_spaces/configs/robot_configs.py 
122
123
124
125
126
127
128
def get_robot_dir(self) -> Path:
    """
    Get the path to the robot directory, which may or may not be a prepackaged MlSpaces robot.
    """
    if self.robot_dir is not None:
        return self.robot_dir
    return get_robot_path(self.name)
get_robot_xml_path 
get_robot_xml_path() -> Path

Get the full path to the robot XML file.

Source code in molmo_spaces/configs/robot_configs.py 
130
131
132
133
134
def get_robot_xml_path(self) -> Path:
    """
    Get the full path to the robot XML file.
    """
    return self.get_robot_dir() / self.robot_xml_path
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(_context)

Ensure action_noise_config is always initialized, even when loading from old configs.

Source code in molmo_spaces/configs/robot_configs.py 
311
312
313
def model_post_init(self, _context):
    super().model_post_init(_context)
    self.robot_view_factory = partial(RBY1RobotView, holo_base=self.use_holo_base)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

task_configs

Task configuration classes for MolmoSpaces experiments.

Classes:

Name Description
BaseMujocoTaskConfig

Base configuration for MuJoCo tasks.
DoorOpeningTaskConfig

Configuration for RBY1 door opening task.
NavToObjTaskConfig

Configuration for RBY1 navigation to object task.
OpeningTaskConfig

Configuration for opening task.
PackingTaskConfig
PickAndPlaceColorTaskConfig
PickAndPlaceNextToTaskConfig
PickAndPlaceTaskConfig
PickTaskConfig

Configuration for Franka move-to-pose task.

Attributes:

Name Type Description
AllTaskConfigs TypeAlias

AllTaskConfigs module-attribute 

AllTaskConfigs: TypeAlias = BaseMujocoTaskConfig | PickTaskConfig | PickAndPlaceTaskConfig | PickAndPlaceColorTaskConfig | PickAndPlaceNextToTaskConfig | PackingTaskConfig | OpeningTaskConfig | DoorOpeningTaskConfig | NavToObjTaskConfig

BaseMujocoTaskConfig

Bases: Config

Base configuration for MuJoCo tasks.

NOTE: If these task config parameters are left to None, they will be sampled by the task sampler. If these task config parameters are not None, their value will take precedence over any parameters sampled by the task sampler and will remain fixed across all simulation tasks sampled by the task sampler.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
model_config
object_poses dict[str, list[float]] | None
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
task_cls instance-attribute 
task_cls: type | None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

DoorOpeningTaskConfig

Bases: BaseMujocoTaskConfig

Configuration for RBY1 door opening task.

NOTE: If these task config parameters are left to None, they will be sampled by the task sampler. If these task config parameters are not None, their value will take precedence over any parameters sampled by the task sampler and will remain fixed across all simulation tasks sampled by the task sampler.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
additional_tcp_offset_distance float
additional_tcp_rotation_offset_mat ndarray
articulated_joint_range ndarray | None
articulated_joint_reset_state ndarray | None
door_body_name str | None
door_open_reward float
door_openness_threshold float
model_config
object_poses dict[str, list[float]] | None
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
viz_target_ee bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
additional_tcp_offset_distance class-attribute instance-attribute 
additional_tcp_offset_distance: float = 0.03
additional_tcp_rotation_offset_mat class-attribute instance-attribute 
additional_tcp_rotation_offset_mat: ndarray = R.from_euler('Y', -90, degrees=True).as_matrix()
articulated_joint_range class-attribute instance-attribute 
articulated_joint_range: ndarray | None = None
articulated_joint_reset_state class-attribute instance-attribute 
articulated_joint_reset_state: ndarray | None = None
door_body_name class-attribute instance-attribute 
door_body_name: str | None = None
door_open_reward class-attribute instance-attribute 
door_open_reward: float = 1.0
door_openness_threshold class-attribute instance-attribute 
door_openness_threshold: float = 0.67
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
task_cls class-attribute instance-attribute 
task_cls: type | None = None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
viz_target_ee class-attribute instance-attribute 
viz_target_ee: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

NavToObjTaskConfig

Bases: BaseMujocoTaskConfig

Configuration for RBY1 navigation to object task.

NOTE: If these task config parameters are left to None, they will be sampled by the task sampler. If these task config parameters are not None, their value will take precedence over any parameters sampled by the task sampler and will remain fixed across all simulation tasks sampled by the task sampler. Uses pickup_obj_name for compatibility with EvalTaskSampler.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
enable_rendering bool
model_config
object_poses dict[str, list[float]] | None
pickup_obj_candidates list[str] | None
pickup_obj_category str | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
pickup_obj_synset str | None
receptacle_name str | None
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
seed int | None
succ_pos_threshold float
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
enable_rendering class-attribute instance-attribute 
enable_rendering: bool = True
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
pickup_obj_candidates class-attribute instance-attribute 
pickup_obj_candidates: list[str] | None = None
pickup_obj_category class-attribute instance-attribute 
pickup_obj_category: str | None = None
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_obj_start_pose class-attribute instance-attribute 
pickup_obj_start_pose: list[float] | None = None
pickup_obj_synset class-attribute instance-attribute 
pickup_obj_synset: str | None = None
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
seed class-attribute instance-attribute 
seed: int | None = None
succ_pos_threshold class-attribute instance-attribute 
succ_pos_threshold: float = 1.5
task_cls class-attribute instance-attribute 
task_cls: type | None = None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

OpeningTaskConfig

Bases: PickTaskConfig

Configuration for opening task.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
any_inst_of_category bool
articulation_object_name str | None
enable_rendering bool
joint_goal_position float | None
joint_index int | None
joint_name str | None
joint_start_position float | None
model_config
object_poses dict[str, list[float]] | None
pickup_obj_goal_pose list[float] | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
place_target_name str | None
receptacle_name str | None
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
succ_pos_threshold float
task_cls type | None
task_success_threshold float
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
any_inst_of_category class-attribute instance-attribute 
any_inst_of_category: bool = False
articulation_object_name class-attribute instance-attribute 
articulation_object_name: str | None = None
enable_rendering class-attribute instance-attribute 
enable_rendering: bool = True
joint_goal_position class-attribute instance-attribute 
joint_goal_position: float | None = None
joint_index class-attribute instance-attribute 
joint_index: int | None = 0
joint_name class-attribute instance-attribute 
joint_name: str | None = None
joint_start_position class-attribute instance-attribute 
joint_start_position: float | None = None
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
pickup_obj_goal_pose class-attribute instance-attribute 
pickup_obj_goal_pose: list[float] | None = None
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_obj_start_pose class-attribute instance-attribute 
pickup_obj_start_pose: list[float] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
succ_pos_threshold class-attribute instance-attribute 
succ_pos_threshold: float = 0.01
task_cls class-attribute instance-attribute 
task_cls: type | None = None
task_success_threshold class-attribute instance-attribute 
task_success_threshold: float = 0.2
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PackingTaskConfig

Bases: PickAndPlaceTaskConfig

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
carry_forward_rel_pos_threshold float
carry_forward_rel_rot_threshold float
enable_rendering bool
max_place_receptacle_pos_displacement float
max_place_receptacle_rot_displacement float
model_config
object_poses dict[str, list[float]] | None
pickup_obj_goal_pose list[float] | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
place_receptacle_name str | None
place_receptacle_start_pose list[float] | None
place_target_name str | None
receptacle_name str | None
receptacle_supported_weight_frac float
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
succ_pos_threshold float
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
carry_forward_rel_pos_threshold class-attribute instance-attribute 
carry_forward_rel_pos_threshold: float = 0.005
carry_forward_rel_rot_threshold class-attribute instance-attribute 
carry_forward_rel_rot_threshold: float = np.radians(10)
enable_rendering class-attribute instance-attribute 
enable_rendering: bool = True
max_place_receptacle_pos_displacement class-attribute instance-attribute 
max_place_receptacle_pos_displacement: float = 0.1
max_place_receptacle_rot_displacement class-attribute instance-attribute 
max_place_receptacle_rot_displacement: float = np.radians(45)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
pickup_obj_goal_pose class-attribute instance-attribute 
pickup_obj_goal_pose: list[float] | None = None
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_obj_start_pose class-attribute instance-attribute 
pickup_obj_start_pose: list[float] | None = None
place_receptacle_name class-attribute instance-attribute 
place_receptacle_name: str | None = None
place_receptacle_start_pose class-attribute instance-attribute 
place_receptacle_start_pose: list[float] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_supported_weight_frac class-attribute instance-attribute 
receptacle_supported_weight_frac: float = 0.5
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
succ_pos_threshold class-attribute instance-attribute 
succ_pos_threshold: float = np.inf
task_cls class-attribute instance-attribute 
task_cls: type | None = None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceColorTaskConfig

Bases: PickAndPlaceTaskConfig

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
carry_forward_rel_pos_threshold float
carry_forward_rel_rot_threshold float
enable_rendering bool
max_place_receptacle_pos_displacement float
max_place_receptacle_rot_displacement float
model_config
object_colors dict[str, list[float]] | None
object_poses dict[str, list[float]] | None
other_receptacle_names list[str] | None
other_receptacle_start_poses dict[str, list[float]] | None
pickup_obj_goal_pose list[float] | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
place_receptacle_name str | None
place_receptacle_start_pose list[float] | None
place_target_name str | None
receptacle_name str | None
receptacle_supported_weight_frac float
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
succ_pos_threshold float
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
carry_forward_rel_pos_threshold class-attribute instance-attribute 
carry_forward_rel_pos_threshold: float = 0.005
carry_forward_rel_rot_threshold class-attribute instance-attribute 
carry_forward_rel_rot_threshold: float = np.radians(10)
enable_rendering class-attribute instance-attribute 
enable_rendering: bool = True
max_place_receptacle_pos_displacement class-attribute instance-attribute 
max_place_receptacle_pos_displacement: float = 0.1
max_place_receptacle_rot_displacement class-attribute instance-attribute 
max_place_receptacle_rot_displacement: float = np.radians(45)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_colors class-attribute instance-attribute 
object_colors: dict[str, list[float]] | None = None
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
other_receptacle_names class-attribute instance-attribute 
other_receptacle_names: list[str] | None = None
other_receptacle_start_poses class-attribute instance-attribute 
other_receptacle_start_poses: dict[str, list[float]] | None = None
pickup_obj_goal_pose class-attribute instance-attribute 
pickup_obj_goal_pose: list[float] | None = None
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_obj_start_pose class-attribute instance-attribute 
pickup_obj_start_pose: list[float] | None = None
place_receptacle_name class-attribute instance-attribute 
place_receptacle_name: str | None = None
place_receptacle_start_pose class-attribute instance-attribute 
place_receptacle_start_pose: list[float] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_supported_weight_frac class-attribute instance-attribute 
receptacle_supported_weight_frac: float = 0.5
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
succ_pos_threshold class-attribute instance-attribute 
succ_pos_threshold: float = np.inf
task_cls class-attribute instance-attribute 
task_cls: type | None = None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceNextToTaskConfig

Bases: PickAndPlaceTaskConfig

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
carry_forward_rel_pos_threshold float
carry_forward_rel_rot_threshold float
enable_rendering bool
max_place_receptacle_pos_displacement float
max_place_receptacle_rot_displacement float
max_surface_to_surface_gap float
min_surface_to_surface_gap float
model_config
object_poses dict[str, list[float]] | None
pickup_obj_goal_pose list[float] | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
place_receptacle_name str | None
place_receptacle_start_pose list[float] | None
place_target_name str | None
receptacle_name str | None
receptacle_supported_weight_frac float
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
succ_pos_threshold float
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
carry_forward_rel_pos_threshold class-attribute instance-attribute 
carry_forward_rel_pos_threshold: float = 0.005
carry_forward_rel_rot_threshold class-attribute instance-attribute 
carry_forward_rel_rot_threshold: float = np.radians(10)
enable_rendering class-attribute instance-attribute 
enable_rendering: bool = True
max_place_receptacle_pos_displacement class-attribute instance-attribute 
max_place_receptacle_pos_displacement: float = 0.05
max_place_receptacle_rot_displacement class-attribute instance-attribute 
max_place_receptacle_rot_displacement: float = np.radians(45)
max_surface_to_surface_gap class-attribute instance-attribute 
max_surface_to_surface_gap: float = 0.05
min_surface_to_surface_gap class-attribute instance-attribute 
min_surface_to_surface_gap: float = 0
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
pickup_obj_goal_pose class-attribute instance-attribute 
pickup_obj_goal_pose: list[float] | None = None
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_obj_start_pose class-attribute instance-attribute 
pickup_obj_start_pose: list[float] | None = None
place_receptacle_name class-attribute instance-attribute 
place_receptacle_name: str | None = None
place_receptacle_start_pose class-attribute instance-attribute 
place_receptacle_start_pose: list[float] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_supported_weight_frac class-attribute instance-attribute 
receptacle_supported_weight_frac: float = 0.5
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
succ_pos_threshold class-attribute instance-attribute 
succ_pos_threshold: float = np.inf
task_cls class-attribute instance-attribute 
task_cls: type | None = None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceTaskConfig

Bases: PickTaskConfig

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
carry_forward_rel_pos_threshold float
carry_forward_rel_rot_threshold float
enable_rendering bool
max_place_receptacle_pos_displacement float
max_place_receptacle_rot_displacement float
model_config
object_poses dict[str, list[float]] | None
pickup_obj_goal_pose list[float] | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
place_receptacle_name str | None
place_receptacle_start_pose list[float] | None
place_target_name str | None
receptacle_name str | None
receptacle_supported_weight_frac float
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
succ_pos_threshold float
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
carry_forward_rel_pos_threshold class-attribute instance-attribute 
carry_forward_rel_pos_threshold: float = 0.005
carry_forward_rel_rot_threshold class-attribute instance-attribute 
carry_forward_rel_rot_threshold: float = np.radians(10)
enable_rendering class-attribute instance-attribute 
enable_rendering: bool = True
max_place_receptacle_pos_displacement class-attribute instance-attribute 
max_place_receptacle_pos_displacement: float = 0.1
max_place_receptacle_rot_displacement class-attribute instance-attribute 
max_place_receptacle_rot_displacement: float = np.radians(45)
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
pickup_obj_goal_pose class-attribute instance-attribute 
pickup_obj_goal_pose: list[float] | None = None
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_obj_start_pose class-attribute instance-attribute 
pickup_obj_start_pose: list[float] | None = None
place_receptacle_name class-attribute instance-attribute 
place_receptacle_name: str | None = None
place_receptacle_start_pose class-attribute instance-attribute 
place_receptacle_start_pose: list[float] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_supported_weight_frac class-attribute instance-attribute 
receptacle_supported_weight_frac: float = 0.5
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
succ_pos_threshold class-attribute instance-attribute 
succ_pos_threshold: float = np.inf
task_cls class-attribute instance-attribute 
task_cls: type | None = None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickTaskConfig

Bases: BaseMujocoTaskConfig

Configuration for Franka move-to-pose task.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
action_dtype str
added_objects dict[str, Path]
enable_rendering bool
model_config
object_poses dict[str, list[float]] | None
pickup_obj_goal_pose list[float] | None
pickup_obj_name str | None
pickup_obj_start_pose list[float] | None
place_target_name str | None
receptacle_name str | None
referral_expressions dict[str, str]
referral_expressions_priority dict[str, list[tuple[float, float, str]]]
robot_base_pose list[float] | None
succ_pos_threshold float
task_cls type | None
tracked_object_names list[str] | None
use_sensors bool
action_dtype class-attribute instance-attribute 
action_dtype: str = 'float32'
added_objects class-attribute instance-attribute 
added_objects: dict[str, Path] = {}
enable_rendering class-attribute instance-attribute 
enable_rendering: bool = True
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
object_poses class-attribute instance-attribute 
object_poses: dict[str, list[float]] | None = None
pickup_obj_goal_pose class-attribute instance-attribute 
pickup_obj_goal_pose: list[float] | None = None
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_obj_start_pose class-attribute instance-attribute 
pickup_obj_start_pose: list[float] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
referral_expressions class-attribute instance-attribute 
referral_expressions: dict[str, str] = {}
referral_expressions_priority class-attribute instance-attribute 
referral_expressions_priority: dict[str, list[tuple[float, float, str]]] = {}
robot_base_pose class-attribute instance-attribute 
robot_base_pose: list[float] | None = None
succ_pos_threshold class-attribute instance-attribute 
succ_pos_threshold: float = 0.01
task_cls class-attribute instance-attribute 
task_cls: type | None = None
tracked_object_names class-attribute instance-attribute 
tracked_object_names: list[str] | None = None
use_sensors class-attribute instance-attribute 
use_sensors: bool = True
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

task_sampler_configs

Task sampler configuration classes for MolmoSpaces experiments.

Classes:

Name Description
BaseMujocoTaskSamplerConfig

Base configuration for task samplers.
DoorOpeningTaskSamplerConfig

Configuration for RBY1 door opening task sampler.
NavToObjTaskSamplerConfig

Configuration for navigation to object task sampler.
ObjectCentricTaskSamplerConfig
OccupancyMapImpl

Which occupancy-map implementation an env hands back from
OpenTaskSamplerConfig
PackingTaskSamplerConfig
PickAndPlaceColorTaskSamplerConfig

Configuration for pick and place color task sampler.
PickAndPlaceNextToTaskSamplerConfig
PickAndPlaceTaskSamplerConfig
PickTaskSamplerConfig

Configuration for Franka move-to-pose task sampler.
RUMPickTaskSamplerConfig

BaseMujocoTaskSamplerConfig

Bases: Config

Base configuration for task samplers.

A task is sampled based on this configuration.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
check_robot_placement_visibility bool
enable_texture_randomization bool
episodes_per_batch int
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_tasks int | None
max_total_attempts_multiplier int
model_config
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
house_inds instance-attribute 
house_inds: list[int] | None
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_tasks instance-attribute 
max_tasks: int | None
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
samples_per_house instance-attribute 
samples_per_house: int | None
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size instance-attribute 
task_batch_size: int
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

DoorOpeningTaskSamplerConfig

Bases: BaseMujocoTaskSamplerConfig

Configuration for RBY1 door opening task sampler.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
choose_random_door_from_scene bool
dataset_name str
door_damping_range tuple
door_frictionloss_range tuple
door_stiffness_range tuple
enable_door_joint_randomization bool
enable_texture_randomization bool
episodes_per_batch int
fixed_door_name str | None
handle_damping_range tuple
handle_frictionloss_range tuple
handle_stiffness_range tuple
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_tasks float
max_total_attempts_multiplier int
model_config
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
random_seed int | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
robot_base_pose_noise float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (1.0, 1.5)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
choose_random_door_from_scene class-attribute instance-attribute 
choose_random_door_from_scene: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
door_damping_range class-attribute instance-attribute 
door_damping_range: tuple = (8, 12)
door_frictionloss_range class-attribute instance-attribute 
door_frictionloss_range: tuple = (8, 12)
door_stiffness_range class-attribute instance-attribute 
door_stiffness_range: tuple = (3, 7)
enable_door_joint_randomization class-attribute instance-attribute 
enable_door_joint_randomization: bool = True
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
fixed_door_name class-attribute instance-attribute 
fixed_door_name: str | None = None
handle_damping_range class-attribute instance-attribute 
handle_damping_range: tuple = (80, 120)
handle_frictionloss_range class-attribute instance-attribute 
handle_frictionloss_range: tuple = (40, 60)
handle_stiffness_range class-attribute instance-attribute 
handle_stiffness_range: tuple = (200, 300)
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 22))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
random_seed class-attribute instance-attribute 
random_seed: int | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
robot_base_pose_noise class-attribute instance-attribute 
robot_base_pose_noise: float = 0.1
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = 0.25
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.7
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 1
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

NavToObjTaskSamplerConfig

Bases: ObjectCentricTaskSamplerConfig

Configuration for navigation to object task sampler. Uses pickup_types/pickup_obj_name for compatibility with EvalTaskSampler.

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_batch int
face_target bool
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_robot_placement_attempts int
max_tasks float
max_total_attempts_multiplier int
max_valid_candidates int
model_config
objaverse_oversampling_factor int
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
robot_object_z_offset float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (1.0, 10.0)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
face_target class-attribute instance-attribute 
face_target: bool = True
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = []
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
max_valid_candidates class-attribute instance-attribute 
max_valid_candidates: int = 6
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = 0.1
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.3
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 1
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

ObjectCentricTaskSamplerConfig

Bases: BaseMujocoTaskSamplerConfig

Methods:

Name Description
from_dict
load_from_json
save_to_json
to_dict
to_json

Attributes:

Name Type Description
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_batch int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_tasks int | None
max_total_attempts_multiplier int
model_config
objaverse_oversampling_factor int
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds instance-attribute 
house_inds: list[int] | None
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_tasks instance-attribute 
max_tasks: int | None
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
samples_per_house instance-attribute 
samples_per_house: int | None
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size instance-attribute 
task_batch_size: int
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

OccupancyMapImpl

Bases: StrEnum

Which occupancy-map implementation an env hands back from get_occupancy_map().

THOR utils/scene_maps.ProcTHORMap / iTHORMap -- molmo_spaces' own, the default for every task and robot. has a room map (room_ids_to_name, get_free_points_by_room, room-scoped label_at). AABB utils/scene_maps_aabb.AABBMap -- from the FetchMan (g1_molmo) repo. Mostly 99% similar to THORMap, slighlty more permissive in floor labeling and slighly faster.

Attributes:

Name Type Description
AABB
THOR
AABB class-attribute instance-attribute 
AABB = 'aabb'
THOR class-attribute instance-attribute 
THOR = 'thor'

OpenTaskSamplerConfig

Bases: PickTaskSamplerConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_radius_range tuple[float, float]
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
target_initial_state_open_percentage float
task_batch_size int
task_sampler_class type | None
verbose bool
added_pickup_class_max_uids class-attribute instance-attribute 
added_pickup_class_max_uids: int | None = None
added_pickup_class_rank class-attribute instance-attribute 
added_pickup_class_rank: int | list[int] | None = None
added_pickup_namespace class-attribute instance-attribute 
added_pickup_namespace: str = 'pickup/'
added_pickup_objects class-attribute instance-attribute 
added_pickup_objects: list[str] | None = None
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_added_pickup class-attribute instance-attribute 
episodes_per_added_pickup: int = 1
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = ['droid']
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 4))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_added_pickup_placement_attempts class-attribute instance-attribute 
max_added_pickup_placement_attempts: int = 100
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_object_placement_attempts class-attribute instance-attribute 
max_object_placement_attempts: int = 200
max_receptacle_attempts class-attribute instance-attribute 
max_receptacle_attempts: int = 10
max_reference_to_added_pickup_dist class-attribute instance-attribute 
max_reference_to_added_pickup_dist: float = 0.5
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_robot_to_added_pickup_dist class-attribute instance-attribute 
max_robot_to_added_pickup_dist: float = 0.7
max_robot_to_obj_dist class-attribute instance-attribute 
max_robot_to_obj_dist: float = 0.6
max_robot_to_target_dist class-attribute instance-attribute 
max_robot_to_target_dist: float = 0.6
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
min_object_separation class-attribute instance-attribute 
min_object_separation: float = 0.05
min_reference_to_added_pickup_dist class-attribute instance-attribute 
min_reference_to_added_pickup_dist: float = 0.15
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_added_pickups class-attribute instance-attribute 
num_added_pickups: int = 30
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
object_placement_radius_range class-attribute instance-attribute 
object_placement_radius_range: tuple[float, float] = (0.1, 0.8)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
placement_volume_name class-attribute instance-attribute 
placement_volume_name: str | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_types class-attribute instance-attribute 
receptacle_types: list[str] = RECEPTACLE_TYPES_THOR
referral_expression_clip_filter class-attribute instance-attribute 
referral_expression_clip_filter: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = -1.0
robot_object_z_offset_random_max class-attribute instance-attribute 
robot_object_z_offset_random_max: float = 0
robot_object_z_offset_random_min class-attribute instance-attribute 
robot_object_z_offset_random_min: float = 0
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_radius_range class-attribute instance-attribute 
robot_placement_radius_range: tuple[float, float] = (0.3, 0.8)
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = 0.25
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.15
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 2
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
target_initial_state_open_percentage class-attribute instance-attribute 
target_initial_state_open_percentage: float = 0
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PackingTaskSamplerConfig

Bases: PickAndPlaceTaskSamplerConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
box_uids list[str] | None
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
episodes_per_receptacle int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_object_to_receptacle_dist float
max_place_receptacle_sampling_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_place_receptacle_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_object_to_receptacle_dist float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
num_place_receptacles int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_receptacle_namespace str
place_receptacle_types list[str]
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
added_pickup_class_max_uids class-attribute instance-attribute 
added_pickup_class_max_uids: int | None = None
added_pickup_class_rank class-attribute instance-attribute 
added_pickup_class_rank: int | list[int] | None = None
added_pickup_namespace class-attribute instance-attribute 
added_pickup_namespace: str = 'pickup/'
added_pickup_objects class-attribute instance-attribute 
added_pickup_objects: list[str] | None = None
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)
box_uids class-attribute instance-attribute 
box_uids: list[str] | None = None
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_added_pickup class-attribute instance-attribute 
episodes_per_added_pickup: int = 1
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
episodes_per_receptacle class-attribute instance-attribute 
episodes_per_receptacle: int = 2
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 4))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_added_pickup_placement_attempts class-attribute instance-attribute 
max_added_pickup_placement_attempts: int = 100
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_object_placement_attempts class-attribute instance-attribute 
max_object_placement_attempts: int = 200
max_object_to_receptacle_dist class-attribute instance-attribute 
max_object_to_receptacle_dist: float = 0.5
max_place_receptacle_sampling_attempts class-attribute instance-attribute 
max_place_receptacle_sampling_attempts: int = 100
max_receptacle_attempts class-attribute instance-attribute 
max_receptacle_attempts: int = 10
max_reference_to_added_pickup_dist class-attribute instance-attribute 
max_reference_to_added_pickup_dist: float = 0.5
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_robot_to_added_pickup_dist class-attribute instance-attribute 
max_robot_to_added_pickup_dist: float = 0.7
max_robot_to_obj_dist class-attribute instance-attribute 
max_robot_to_obj_dist: float = 0.6
max_robot_to_place_receptacle_dist class-attribute instance-attribute 
max_robot_to_place_receptacle_dist: float = 0.7
max_robot_to_target_dist class-attribute instance-attribute 
max_robot_to_target_dist: float = 0.6
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
min_object_separation class-attribute instance-attribute 
min_object_separation: float = 0.05
min_object_to_receptacle_dist class-attribute instance-attribute 
min_object_to_receptacle_dist: float = 0.15
min_reference_to_added_pickup_dist class-attribute instance-attribute 
min_reference_to_added_pickup_dist: float = 0.15
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_added_pickups class-attribute instance-attribute 
num_added_pickups: int = 30
num_place_receptacles class-attribute instance-attribute 
num_place_receptacles: int = 3
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
object_placement_radius_range class-attribute instance-attribute 
object_placement_radius_range: tuple[float, float] = (0.1, 0.8)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
place_receptacle_namespace class-attribute instance-attribute 
place_receptacle_namespace: str = 'place_receptacle/'
place_receptacle_types class-attribute instance-attribute 
place_receptacle_types: list[str] = []
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
placement_volume_name class-attribute instance-attribute 
placement_volume_name: str | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_types class-attribute instance-attribute 
receptacle_types: list[str] = RECEPTACLE_TYPES_THOR
referral_expression_clip_filter class-attribute instance-attribute 
referral_expression_clip_filter: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = -0.75
robot_object_z_offset_random_max class-attribute instance-attribute 
robot_object_z_offset_random_max: float = 0.25
robot_object_z_offset_random_min class-attribute instance-attribute 
robot_object_z_offset_random_min: float = -0.3
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.15
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 2
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceColorTaskSamplerConfig

Bases: PickAndPlaceTaskSamplerConfig

Configuration for pick and place color task sampler.

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
episodes_per_receptacle int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_object_to_receptacle_dist float
max_place_receptacle_sampling_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_place_receptacle_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_object_to_receptacle_dist float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
num_place_receptacles int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_receptacle_namespace str
place_receptacle_types list[str]
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
added_pickup_class_max_uids class-attribute instance-attribute 
added_pickup_class_max_uids: int | None = None
added_pickup_class_rank class-attribute instance-attribute 
added_pickup_class_rank: int | list[int] | None = None
added_pickup_namespace class-attribute instance-attribute 
added_pickup_namespace: str = 'pickup/'
added_pickup_objects class-attribute instance-attribute 
added_pickup_objects: list[str] | None = None
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_added_pickup class-attribute instance-attribute 
episodes_per_added_pickup: int = 1
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
episodes_per_receptacle class-attribute instance-attribute 
episodes_per_receptacle: int = 2
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 4))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_added_pickup_placement_attempts class-attribute instance-attribute 
max_added_pickup_placement_attempts: int = 100
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_object_placement_attempts class-attribute instance-attribute 
max_object_placement_attempts: int = 200
max_object_to_receptacle_dist class-attribute instance-attribute 
max_object_to_receptacle_dist: float = 0.5
max_place_receptacle_sampling_attempts class-attribute instance-attribute 
max_place_receptacle_sampling_attempts: int = 100
max_receptacle_attempts class-attribute instance-attribute 
max_receptacle_attempts: int = 10
max_reference_to_added_pickup_dist class-attribute instance-attribute 
max_reference_to_added_pickup_dist: float = 0.5
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_robot_to_added_pickup_dist class-attribute instance-attribute 
max_robot_to_added_pickup_dist: float = 0.7
max_robot_to_obj_dist class-attribute instance-attribute 
max_robot_to_obj_dist: float = 0.6
max_robot_to_place_receptacle_dist class-attribute instance-attribute 
max_robot_to_place_receptacle_dist: float = 0.7
max_robot_to_target_dist class-attribute instance-attribute 
max_robot_to_target_dist: float = 0.6
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
min_object_separation class-attribute instance-attribute 
min_object_separation: float = 0.05
min_object_to_receptacle_dist class-attribute instance-attribute 
min_object_to_receptacle_dist: float = 0.15
min_reference_to_added_pickup_dist class-attribute instance-attribute 
min_reference_to_added_pickup_dist: float = 0.15
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_added_pickups class-attribute instance-attribute 
num_added_pickups: int = 30
num_place_receptacles class-attribute instance-attribute 
num_place_receptacles: int = 3
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
object_placement_radius_range class-attribute instance-attribute 
object_placement_radius_range: tuple[float, float] = (0.1, 0.8)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
place_receptacle_namespace class-attribute instance-attribute 
place_receptacle_namespace: str = 'place_receptacle/'
place_receptacle_types class-attribute instance-attribute 
place_receptacle_types: list[str] = []
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
placement_volume_name class-attribute instance-attribute 
placement_volume_name: str | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_types class-attribute instance-attribute 
receptacle_types: list[str] = RECEPTACLE_TYPES_THOR
referral_expression_clip_filter class-attribute instance-attribute 
referral_expression_clip_filter: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = -0.75
robot_object_z_offset_random_max class-attribute instance-attribute 
robot_object_z_offset_random_max: float = 0.25
robot_object_z_offset_random_min class-attribute instance-attribute 
robot_object_z_offset_random_min: float = -0.3
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.15
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 2
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceNextToTaskSamplerConfig

Bases: PickAndPlaceTaskSamplerConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
episodes_per_receptacle int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_object_to_receptacle_dist float
max_place_receptacle_sampling_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_place_receptacle_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_object_to_receptacle_dist float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
num_place_receptacles int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_receptacle_namespace str
place_receptacle_types list[str]
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
added_pickup_class_max_uids class-attribute instance-attribute 
added_pickup_class_max_uids: int | None = None
added_pickup_class_rank class-attribute instance-attribute 
added_pickup_class_rank: int | list[int] | None = None
added_pickup_namespace class-attribute instance-attribute 
added_pickup_namespace: str = 'pickup/'
added_pickup_objects class-attribute instance-attribute 
added_pickup_objects: list[str] | None = None
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_added_pickup class-attribute instance-attribute 
episodes_per_added_pickup: int = 1
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
episodes_per_receptacle class-attribute instance-attribute 
episodes_per_receptacle: int = 0
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 4))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_added_pickup_placement_attempts class-attribute instance-attribute 
max_added_pickup_placement_attempts: int = 100
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_object_placement_attempts class-attribute instance-attribute 
max_object_placement_attempts: int = 200
max_object_to_receptacle_dist class-attribute instance-attribute 
max_object_to_receptacle_dist: float = 0.5
max_place_receptacle_sampling_attempts class-attribute instance-attribute 
max_place_receptacle_sampling_attempts: int = 100
max_receptacle_attempts class-attribute instance-attribute 
max_receptacle_attempts: int = 10
max_reference_to_added_pickup_dist class-attribute instance-attribute 
max_reference_to_added_pickup_dist: float = 0.5
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_robot_to_added_pickup_dist class-attribute instance-attribute 
max_robot_to_added_pickup_dist: float = 0.7
max_robot_to_obj_dist class-attribute instance-attribute 
max_robot_to_obj_dist: float = 0.6
max_robot_to_place_receptacle_dist class-attribute instance-attribute 
max_robot_to_place_receptacle_dist: float = 0.7
max_robot_to_target_dist class-attribute instance-attribute 
max_robot_to_target_dist: float = 0.6
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
min_object_separation class-attribute instance-attribute 
min_object_separation: float = 0.05
min_object_to_receptacle_dist class-attribute instance-attribute 
min_object_to_receptacle_dist: float = 0.3
min_reference_to_added_pickup_dist class-attribute instance-attribute 
min_reference_to_added_pickup_dist: float = 0.15
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_added_pickups class-attribute instance-attribute 
num_added_pickups: int = 30
num_place_receptacles class-attribute instance-attribute 
num_place_receptacles: int = 3
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
object_placement_radius_range class-attribute instance-attribute 
object_placement_radius_range: tuple[float, float] = (0.1, 0.8)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
place_receptacle_namespace class-attribute instance-attribute 
place_receptacle_namespace: str = 'place_receptacle/'
place_receptacle_types class-attribute instance-attribute 
place_receptacle_types: list[str] = []
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
placement_volume_name class-attribute instance-attribute 
placement_volume_name: str | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_types class-attribute instance-attribute 
receptacle_types: list[str] = RECEPTACLE_TYPES_THOR
referral_expression_clip_filter class-attribute instance-attribute 
referral_expression_clip_filter: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = -0.75
robot_object_z_offset_random_max class-attribute instance-attribute 
robot_object_z_offset_random_max: float = 0.25
robot_object_z_offset_random_min class-attribute instance-attribute 
robot_object_z_offset_random_min: float = -0.3
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.15
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 2
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickAndPlaceTaskSamplerConfig

Bases: PickTaskSamplerConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
episodes_per_receptacle int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_object_to_receptacle_dist float
max_place_receptacle_sampling_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_place_receptacle_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_object_to_receptacle_dist float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
num_place_receptacles int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_receptacle_namespace str
place_receptacle_types list[str]
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
added_pickup_class_max_uids class-attribute instance-attribute 
added_pickup_class_max_uids: int | None = None
added_pickup_class_rank class-attribute instance-attribute 
added_pickup_class_rank: int | list[int] | None = None
added_pickup_namespace class-attribute instance-attribute 
added_pickup_namespace: str = 'pickup/'
added_pickup_objects class-attribute instance-attribute 
added_pickup_objects: list[str] | None = None
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_added_pickup class-attribute instance-attribute 
episodes_per_added_pickup: int = 1
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
episodes_per_receptacle class-attribute instance-attribute 
episodes_per_receptacle: int = 2
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 4))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_added_pickup_placement_attempts class-attribute instance-attribute 
max_added_pickup_placement_attempts: int = 100
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_object_placement_attempts class-attribute instance-attribute 
max_object_placement_attempts: int = 200
max_object_to_receptacle_dist class-attribute instance-attribute 
max_object_to_receptacle_dist: float = 0.5
max_place_receptacle_sampling_attempts class-attribute instance-attribute 
max_place_receptacle_sampling_attempts: int = 100
max_receptacle_attempts class-attribute instance-attribute 
max_receptacle_attempts: int = 10
max_reference_to_added_pickup_dist class-attribute instance-attribute 
max_reference_to_added_pickup_dist: float = 0.5
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_robot_to_added_pickup_dist class-attribute instance-attribute 
max_robot_to_added_pickup_dist: float = 0.7
max_robot_to_obj_dist class-attribute instance-attribute 
max_robot_to_obj_dist: float = 0.6
max_robot_to_place_receptacle_dist class-attribute instance-attribute 
max_robot_to_place_receptacle_dist: float = 0.7
max_robot_to_target_dist class-attribute instance-attribute 
max_robot_to_target_dist: float = 0.6
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
min_object_separation class-attribute instance-attribute 
min_object_separation: float = 0.05
min_object_to_receptacle_dist class-attribute instance-attribute 
min_object_to_receptacle_dist: float = 0.15
min_reference_to_added_pickup_dist class-attribute instance-attribute 
min_reference_to_added_pickup_dist: float = 0.15
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_added_pickups class-attribute instance-attribute 
num_added_pickups: int = 30
num_place_receptacles class-attribute instance-attribute 
num_place_receptacles: int = 3
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
object_placement_radius_range class-attribute instance-attribute 
object_placement_radius_range: tuple[float, float] = (0.1, 0.8)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
place_receptacle_namespace class-attribute instance-attribute 
place_receptacle_namespace: str = 'place_receptacle/'
place_receptacle_types class-attribute instance-attribute 
place_receptacle_types: list[str] = []
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
placement_volume_name class-attribute instance-attribute 
placement_volume_name: str | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_types class-attribute instance-attribute 
receptacle_types: list[str] = RECEPTACLE_TYPES_THOR
referral_expression_clip_filter class-attribute instance-attribute 
referral_expression_clip_filter: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = -0.75
robot_object_z_offset_random_max class-attribute instance-attribute 
robot_object_z_offset_random_max: float = 0.25
robot_object_z_offset_random_min class-attribute instance-attribute 
robot_object_z_offset_random_min: float = -0.3
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.15
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 2
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

PickTaskSamplerConfig

Bases: ObjectCentricTaskSamplerConfig

Configuration for Franka move-to-pose task sampler.

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
added_pickup_class_max_uids class-attribute instance-attribute 
added_pickup_class_max_uids: int | None = None
added_pickup_class_rank class-attribute instance-attribute 
added_pickup_class_rank: int | list[int] | None = None
added_pickup_namespace class-attribute instance-attribute 
added_pickup_namespace: str = 'pickup/'
added_pickup_objects class-attribute instance-attribute 
added_pickup_objects: list[str] | None = None
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_added_pickup class-attribute instance-attribute 
episodes_per_added_pickup: int = 1
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 4))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_added_pickup_placement_attempts class-attribute instance-attribute 
max_added_pickup_placement_attempts: int = 100
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_object_placement_attempts class-attribute instance-attribute 
max_object_placement_attempts: int = 200
max_receptacle_attempts class-attribute instance-attribute 
max_receptacle_attempts: int = 10
max_reference_to_added_pickup_dist class-attribute instance-attribute 
max_reference_to_added_pickup_dist: float = 0.5
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_robot_to_added_pickup_dist class-attribute instance-attribute 
max_robot_to_added_pickup_dist: float = 0.7
max_robot_to_obj_dist class-attribute instance-attribute 
max_robot_to_obj_dist: float = 0.6
max_robot_to_target_dist class-attribute instance-attribute 
max_robot_to_target_dist: float = 0.6
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
min_object_separation class-attribute instance-attribute 
min_object_separation: float = 0.05
min_reference_to_added_pickup_dist class-attribute instance-attribute 
min_reference_to_added_pickup_dist: float = 0.15
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_added_pickups class-attribute instance-attribute 
num_added_pickups: int = 30
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
object_placement_radius_range class-attribute instance-attribute 
object_placement_radius_range: tuple[float, float] = (0.1, 0.8)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
placement_volume_name class-attribute instance-attribute 
placement_volume_name: str | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_types class-attribute instance-attribute 
receptacle_types: list[str] = RECEPTACLE_TYPES_THOR
referral_expression_clip_filter class-attribute instance-attribute 
referral_expression_clip_filter: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = -0.75
robot_object_z_offset_random_max class-attribute instance-attribute 
robot_object_z_offset_random_max: float = 0.25
robot_object_z_offset_random_min class-attribute instance-attribute 
robot_object_z_offset_random_min: float = -0.3
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.15
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 2
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")

RUMPickTaskSamplerConfig

Bases: PickTaskSamplerConfig

Methods:

Name Description
from_dict
load_from_json
model_post_init
save_to_json
to_dict
to_json

Attributes:

Name Type Description
added_pickup_class_max_uids int | None
added_pickup_class_rank int | list[int] | None
added_pickup_namespace str
added_pickup_objects list[str] | None
base_pose_sampling_radius_range tuple[float, float]
check_robot_placement_visibility bool
dataset_name str
enable_texture_randomization bool
episodes_per_added_pickup int
episodes_per_batch int
filter_for_grasps bool
grasp_libraries list[str] | None
house_inds list[int] | None
house_variant str
license_policy LicensePolicy
max_added_pickup_placement_attempts int
max_allowed_sequential_irrecoverable_failures int
max_allowed_sequential_rollout_failures int
max_allowed_sequential_task_sampler_failures int
max_asset_failures int
max_object_placement_attempts int
max_receptacle_attempts int
max_reference_to_added_pickup_dist float
max_robot_placement_attempts int
max_robot_to_added_pickup_dist float
max_robot_to_obj_dist float
max_robot_to_target_dist float
max_tasks float
max_total_attempts_multiplier int
min_object_separation float
min_reference_to_added_pickup_dist float
model_config
num_added_pickups int
objaverse_oversampling_factor int
object_placement_radius_range tuple[float, float]
occupancy_map_cache_size int
occupancy_map_impl OccupancyMapImpl
pickup_obj_name str | None
pickup_types list[str] | None
place_target_name str | None
placement_volume_name str | None
randomize_dynamics bool
randomize_lighting bool
randomize_robot_textures bool
randomize_textures bool
randomize_textures_all bool
receptacle_name str | None
receptacle_types list[str]
referral_expression_clip_filter bool
robot_object_z_offset float
robot_object_z_offset_random_max float
robot_object_z_offset_random_min float
robot_placement_exclusion_threshold float
robot_placement_rotation_range_rad float
robot_safety_radius float
samples_per_house int | None
scene_xml_paths list[str] | None
sim_settle_timesteps int
task_batch_size int
task_sampler_class type | None
verbose bool
added_pickup_class_max_uids class-attribute instance-attribute 
added_pickup_class_max_uids: int | None = None
added_pickup_class_rank class-attribute instance-attribute 
added_pickup_class_rank: int | list[int] | None = None
added_pickup_namespace class-attribute instance-attribute 
added_pickup_namespace: str = 'pickup/'
added_pickup_objects class-attribute instance-attribute 
added_pickup_objects: list[str] | None = None
base_pose_sampling_radius_range class-attribute instance-attribute 
base_pose_sampling_radius_range: tuple[float, float] = (0.0, 0.7)
check_robot_placement_visibility class-attribute instance-attribute 
check_robot_placement_visibility: bool = True
dataset_name class-attribute instance-attribute 
dataset_name: str = 'procthor-10k'
enable_texture_randomization class-attribute instance-attribute 
enable_texture_randomization: bool = False
episodes_per_added_pickup class-attribute instance-attribute 
episodes_per_added_pickup: int = 1
episodes_per_batch class-attribute instance-attribute 
episodes_per_batch: int = 4
filter_for_grasps class-attribute instance-attribute 
filter_for_grasps: bool = True
grasp_libraries class-attribute instance-attribute 
grasp_libraries: list[str] | None = None
house_inds class-attribute instance-attribute 
house_inds: list[int] | None = list(range(0, 4))
house_variant class-attribute instance-attribute 
house_variant: str = 'ceiling'
license_policy class-attribute instance-attribute 
license_policy: LicensePolicy = LicensePolicy.NONE
max_added_pickup_placement_attempts class-attribute instance-attribute 
max_added_pickup_placement_attempts: int = 100
max_allowed_sequential_irrecoverable_failures class-attribute instance-attribute 
max_allowed_sequential_irrecoverable_failures: int = 5
max_allowed_sequential_rollout_failures class-attribute instance-attribute 
max_allowed_sequential_rollout_failures: int = 10
max_allowed_sequential_task_sampler_failures class-attribute instance-attribute 
max_allowed_sequential_task_sampler_failures: int = 10
max_asset_failures class-attribute instance-attribute 
max_asset_failures: int = 10
max_object_placement_attempts class-attribute instance-attribute 
max_object_placement_attempts: int = 200
max_receptacle_attempts class-attribute instance-attribute 
max_receptacle_attempts: int = 10
max_reference_to_added_pickup_dist class-attribute instance-attribute 
max_reference_to_added_pickup_dist: float = 0.5
max_robot_placement_attempts class-attribute instance-attribute 
max_robot_placement_attempts: int = 10
max_robot_to_added_pickup_dist class-attribute instance-attribute 
max_robot_to_added_pickup_dist: float = 0.7
max_robot_to_obj_dist class-attribute instance-attribute 
max_robot_to_obj_dist: float = 0.6
max_robot_to_target_dist class-attribute instance-attribute 
max_robot_to_target_dist: float = 0.6
max_tasks class-attribute instance-attribute 
max_tasks: float = math.inf
max_total_attempts_multiplier class-attribute instance-attribute 
max_total_attempts_multiplier: int = 6
min_object_separation class-attribute instance-attribute 
min_object_separation: float = 0.05
min_reference_to_added_pickup_dist class-attribute instance-attribute 
min_reference_to_added_pickup_dist: float = 0.15
model_config class-attribute instance-attribute 
model_config = ConfigDict(arbitrary_types_allowed=True)
num_added_pickups class-attribute instance-attribute 
num_added_pickups: int = 30
objaverse_oversampling_factor class-attribute instance-attribute 
objaverse_oversampling_factor: int = 30
object_placement_radius_range class-attribute instance-attribute 
object_placement_radius_range: tuple[float, float] = (0.1, 0.8)
occupancy_map_cache_size class-attribute instance-attribute 
occupancy_map_cache_size: int = 4
occupancy_map_impl class-attribute instance-attribute 
occupancy_map_impl: OccupancyMapImpl = OccupancyMapImpl.THOR
pickup_obj_name class-attribute instance-attribute 
pickup_obj_name: str | None = None
pickup_types class-attribute instance-attribute 
pickup_types: list[str] | None = None
place_target_name class-attribute instance-attribute 
place_target_name: str | None = None
placement_volume_name class-attribute instance-attribute 
placement_volume_name: str | None = None
randomize_dynamics class-attribute instance-attribute 
randomize_dynamics: bool = False
randomize_lighting class-attribute instance-attribute 
randomize_lighting: bool = False
randomize_robot_textures class-attribute instance-attribute 
randomize_robot_textures: bool = False
randomize_textures class-attribute instance-attribute 
randomize_textures: bool = False
randomize_textures_all class-attribute instance-attribute 
randomize_textures_all: bool = False
receptacle_name class-attribute instance-attribute 
receptacle_name: str | None = None
receptacle_types class-attribute instance-attribute 
receptacle_types: list[str] = RECEPTACLE_TYPES_THOR
referral_expression_clip_filter class-attribute instance-attribute 
referral_expression_clip_filter: bool = False
robot_object_z_offset class-attribute instance-attribute 
robot_object_z_offset: float = 0
robot_object_z_offset_random_max class-attribute instance-attribute 
robot_object_z_offset_random_max: float = 0
robot_object_z_offset_random_min class-attribute instance-attribute 
robot_object_z_offset_random_min: float = 0
robot_placement_exclusion_threshold class-attribute instance-attribute 
robot_placement_exclusion_threshold: float = 0.15
robot_placement_rotation_range_rad class-attribute instance-attribute 
robot_placement_rotation_range_rad: float = math.radians(45)
robot_safety_radius class-attribute instance-attribute 
robot_safety_radius: float = 0.15
samples_per_house class-attribute instance-attribute 
samples_per_house: int | None = 2
scene_xml_paths class-attribute instance-attribute 
scene_xml_paths: list[str] | None = None
sim_settle_timesteps class-attribute instance-attribute 
sim_settle_timesteps: int = 500
task_batch_size class-attribute instance-attribute 
task_batch_size: int = 1
task_sampler_class class-attribute instance-attribute 
task_sampler_class: type | None = None
verbose class-attribute instance-attribute 
verbose: bool = False
from_dict classmethod 
from_dict(data: dict) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
29
30
31
@classmethod
def from_dict(cls, data: dict) -> Config:
    return cls.model_validate(data)
load_from_json classmethod 
load_from_json(file_path: str) -> Config
Source code in molmo_spaces/configs/abstract_config.py 
33
34
35
36
37
@classmethod
def load_from_json(cls, file_path: str) -> Config:
    with open(file_path) as f:
        data = f.read()
    return cls.model_validate_json(data)
model_post_init 
model_post_init(__context) -> None
Source code in molmo_spaces/configs/task_sampler_configs.py 
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
def model_post_init(self, __context) -> None:
    super().model_post_init(__context)
    if self.added_pickup_class_rank is not None:
        from molmo_spaces.utils.synset_utils import _pickupable_class_ranking

        ranking = _pickupable_class_ranking()
        ranks = (
            self.added_pickup_class_rank
            if isinstance(self.added_pickup_class_rank, list)
            else [self.added_pickup_class_rank]
        )
        all_uids: list[str] = []
        for rank in ranks:
            idx = rank - 1
            if idx < 0 or idx >= len(ranking):
                raise ValueError(
                    f"added_pickup_class_rank={rank} out of range [1, {len(ranking)}]"
                )
            uids = ranking[idx][1]
            if self.added_pickup_class_max_uids is not None:
                uids = uids[: self.added_pickup_class_max_uids]
            all_uids.extend(uids)
        self.added_pickup_objects = all_uids
    if self.added_pickup_objects:
        self.objaverse_oversampling_factor = 1
save_to_json 
save_to_json(file_path: str) -> None
Source code in molmo_spaces/configs/abstract_config.py 
39
40
41
def save_to_json(self, file_path: str) -> None:
    with open(file_path, "w") as f:
        f.write(self.to_json())
to_dict 
to_dict() -> dict
Source code in molmo_spaces/configs/abstract_config.py 
22
23
def to_dict(self) -> dict:
    return self.model_dump()
to_json 
to_json() -> str
Source code in molmo_spaces/configs/abstract_config.py 
25
26
27
def to_json(self) -> str:
    # TODO(max): this can cause errors, try printing out the config to see missmatches w/ print(self)
    return self.model_dump_json(warnings="error")