Tooling · Post 4 of 6

A live 3D robot model inside Rerun

Our --display_urdf_path extension to lerobot-record.

LeRobot can stream camera feeds and joint values to Rerun while recording (--display_data=true, described in the imitation learning guide). Numbers and camera images are useful, but during a demonstration it is much easier to judge a motion by watching a 3D model of the arm. So we added one.

What we added

A new option on lerobot-record:

--display_urdf_path=assets/so101/so101_new_calib.urdf

When set together with --display_data=true, the recorder renders the SO-101 in the Rerun viewer and moves it with the live joint readings.

How it works

1. Once at start-up: init_rerun_urdf() loads the URDF with yourdfpy and logs every link's mesh to Rerun as static geometry under world/urdf. Meshes are heavy, so they are sent only once.
2. Every control step: log_rerun_urdf() takes the joint positions from the observation (the keys ending in .pos, in degrees), converts them to radians, updates the URDF configuration, and logs only a small Transform3D per link.

The changes touch visualization_utils.py and lerobot_record.py. The dependency is optional and installed through a new urdf-viz extra in pyproject.toml.