SO-Frame is a cheap, open evaluation frame for SO101 arms (with the LeSlider add-on), built at LiveKit as a reproducible environment for debugging our robotics products: Portal and Agents. It ships with a simulation (URDF + MuJoCo + USD) and a complementary reinforcement-learning pick-and-place task, solved two ways (state-based and vision-based) built on it.

Real	RL in Sim

• Bill of Materials
• CAD (Onshape)
• Simulation (URDF + MuJoCo + USD)
• Reinforcement Learning

The box frame is built entirely from 2020 (20×20 mm) T-slot aluminium extrusion. The LeSlider add-on provides the 1-DOF linear motion and has its own BOM.

Amazon links below are examples (US) to show the right part. Brands and pack sizes vary and listings change, so double-check specs before buying.

Total: 9.5 m of 2020 extrusion (5 × 1 m + 9 × 0.5 m). Extrusion is usually sold in fixed lengths or cut-to-order, so buy to match or cut longer stock down.

Every bracket/handle screw pairs with a matching drop-in T-nut. Use M5 or M4 to match your bracket & T-nut kit. 2020 kits are usually M5 (some use M4); the counts are the same either way. An assortment kit like this M5 T-nut + button-head screw set covers all of the below.

Verify against your kit. Fasteners aren't all modelled in the URDF, so these are the typical counts: hidden 3-way brackets are assumed at 3 bolts + 3 T-nuts each. Some 3-way brackets instead use 6 bolts, or grub/set screws that thread into the extrusion end (no T-nut). Adjust to whatever hardware ships with your brackets.

The flat panels that skin the frame (the matte side panels in the renders) are CNC-cut from white mica sheet. Each DXF below is one cut, so cut one of each except the short side panel, which is cut twice.

The sliding carriage that carries the arm is the LeSlider mechanism (V-wheel gantry + rack & pinion). It adds its own hardware: 4× V-wheel assemblies, 4× M5×25 low-profile screws, 4× M5 nylock nuts, eccentric spacers, and the pinion/rack. See the LeSlider BOM.

The SO-Frame + SO-101 + two cameras, described three ways that share the same meshes: a URDF, a MuJoCo (MJCF) model, and a USD scene. The arm mounts on the frame's slider, with a wrist camera and an overhead camera.

simulation/urdf/so101_on_frame.urdf is the combined model for URDF viewers, PyBullet, Isaac, etc. It includes the slider joint, the arm, and both camera frames (frame_wrist_camera, frame_overhead_camera). See simulation/urdf/README.md for kinematics, joint limits, the mounting details, and the interactive camera-alignment helper.

simulation/mjcf/scene.xml is the MuJoCo model. On top of the URDF geometry it adds actuators, box collisions for the frame, a floor/light/skybox, and the two cameras as real renderable MuJoCo cameras. Load it with:

python -m mujoco.viewer --mjcf=simulation/mjcf/scene.xml

See simulation/mjcf/README.md for actuators, collision, and camera details.

Same setup, plus what each camera sees:

Setup	frame_wrist_camera	frame_overhead_camera

simulation/usd/so101_on_frame.usd is a USD scene with physically-based materials (aluminium, matte mica side panels, white/orange PLA, black plastic, steel) and soft overhead lighting, for usdview / Blender / Omniverse. See simulation/usd/README.md.

Setup	frame_wrist_camera	frame_overhead_camera

rl/ holds the training environments, the shared policy definition, and the deploy stack:

rl/environments/maniskill/   vision-based RL (camera pixels only)  -- feeds deploy
rl/environments/mjlab/       state-based RL (ground-truth poses)
rl/policy/                   the encoder, actor and checkpoint format
rl/deploy/                   run a checkpoint on the physical rig

Both environments implement the same pick-up-a-cube-and-place-it-in-a-bin task, with the cube and bin randomized each episode; they differ in what the policy observes. See rl/README.md for how the four fit together.

Built on mjlab (Isaac Lab's manager-based API on GPU-accelerated MuJoCo-Warp), using the simulation MJCF model. Trains with PPO (rsl-rl) across thousands of parallel environments on ground-truth cube/bin poses.

Heads up on the current policy. It doesn't actually pick and place. The policy found a shortcut and instead putts the cube like a golf shot, whacking it across the workspace and into the bin rather than grasping and lifting it. It's a fun bit of reward hacking, and the reward shaping is still being tuned to coax out a proper grasp.

It's a uv project. From rl/environments/mjlab/:

uv sync
uv run soframe-train Mjlab-Pick-Place-Bin-SO101
uv run soframe-play  Mjlab-Pick-Place-Bin-SO101 --checkpoint-file <path>

Training parameters live in rl/environments/mjlab/train.toml. Because it runs on MuJoCo-Warp, training needs a Linux + NVIDIA GPU machine (macOS can build and CPU smoke-test). See rl/environments/mjlab/README.md for the full environment, reward, curriculum, manager, and config details.

Chained rollouts (wrist camera left, overhead camera right), rendered in the flat shading they were trained on.

Trains purely from the frame's own wrist camera: no ground-truth cube/bin poses, just RGB pixels and proprioception. Built on ManiSkill3 (SAPIEN + PhysX, GPU-parallel), implementing Squint: Fast Visual Reinforcement Learning for Sim-to-Real Robotics (Almuzairee & Christensen, 2026), a visual Soft Actor-Critic that reaches strong success rates in minutes of wall-clock time.

This folder is a direct port of the paper's reference implementation (which already targets an SO-101 arm in ManiSkill3), retargeted onto this repo's frame-mounted rig and its existing calibrated frame_wrist_camera/frame_overhead_camera mounts and simulation/urdf/so101_on_frame.urdf.

SAPIEN vs MuJoCo camera convention: the two engines define a camera's local forward/right/up axes differently. SAPIEN uses (forward, right, up) = (+X, -Y, +Z) (see sapien_utils.look_at's docstring), while MuJoCo uses (forward, right, up) = (-Z, +X, +Y). Both simulation/urdf/so101_on_frame.urdf and simulation/mjcf/so101_on_frame.xml calibrate their camera joints/bodies from the same physical mount, at the same position, but the URDF's rotation is the MJCF's rotation converted through this fixed axis remap (a constant rotation P with R_sapien = R_mujoco @ P), not a copy of the MJCF's raw quaternion: porting the quaternion directly renders the wrong direction in SAPIEN despite the shared calibration.

It's a uv project. From rl/environments/maniskill/:

uv sync
uv run python examples/visualize_sim.py
uv run python train.py                      # squint CNN
uv run python train.py --encoder dino_patch # frozen DINOv2 + patch head

Task, robot and reward constants live in rl/environments/maniskill/src/soframe_rl_maniskill/config.py. The encoder, actor and checkpoint format are shared with deploy in policy/.

Needs a Linux + NVIDIA GPU machine (ManiSkill3/SAPIEN + CUDA; macOS can read/edit code but not train). See rl/environments/maniskill/README.md for the task, observation/reward design, domain randomization, and training details.