E1 · With gantry
5 minRun the boxing replay with the pelvis on the gantry.
Done when: python3 kit/reproduzir.py ~/fth/movimentos/boxe_g1.npz | tail -2 → erro médio de junta: 0.3° and queda: não.
Lesson 4 · 15 min
A policy that imitates any motion
Having the motion in the robot's format is not the same as making it perform it. If you send only the angles to the motors, it imitates for a second and falls: nobody is handling balance. In this lesson you play the boxing in MuJoCo with and without a gantry, measure each joint's error, and see what BeyondMimic, GMT and SONIC solve.
Start the lesson
In this lesson you will
- 1Play a reference with the joints' PD
- 2Tracking error in degrees, per joint
- 3Why following angles doesn't balance the robot
- 4BeyondMimic (now in mjlab), GMT and SONIC: what each one does
Goal: play the motion in MuJoCo with and without a gantry and explain why a tracking policy is needed.
01From reference to standing robot
Click each block.
Click each block of the diagram to see what it does.
Virtual gantry = the base is put back at the reference pose at every step, like a robot hanging in the first test.
02Who solves motion tracking in 2026
Click on each one. All of them train on a GPU; here they are shown, not run.
1. BeyondMimic
Tracks dynamic motions (spinning, running, cartwheel) and uses a guided diffusion policy for tasks without specific training (arXiv 2508.08241, Aug/2025). The whole_body_tracking repository pins Isaac Lab 2.1.0.
As of 10/15/2026, that repository's README says it has been retired: BeyondMimic tracking continues in mjlab, in the Mjlab-Tracking-Flat-Unitree-G1 task.
2. GMT
General tracking with adaptive sampling (easy × hard motions) and a mixture-of-experts layer (arXiv 2506.14770, Jun/2025). Reported by the course research, based on the paper's abstract.
3. SONIC
From NVIDIA: 42M parameters, trained on more than 100M frames (about 700 h). It has three public checkpoints for the G1 (original, low latency, and v1.1 from 07/23/2026), 64-dimension tokens, and a 50 Hz controller.
Deployment in C++ and TensorRT; code under Apache-2.0, weights under the NVIDIA Open Model License (commercial use with attribution).
Format
GMR exports CSV for BeyondMimic, whose imitation pipeline mjlab adopted. Your .npz from lesson 3 is the same data in the kit's own format.
03Replay with the pelvis on the gantry
The joints receive the reference at each 2 ms step; the base is placed back at the reference pose. The error is the difference between the measured angle and the reference angle at the next step.
Terminal · with gantry
F=${FTH:-$HOME/fth}; cd $F/corpo-inteiro && . $F/.venv/bin/activate
python3 kit/reproduzir.py $F/movimentos/boxe_g1.npz; echo "saída: $?"Excerpt · kit/reproduzir.py
if a.modo == "portico":
fth.portico(d, r[:7])
d.ctrl[mot] = r[adr]
mujoco.mj_step(m, d)Example from the repository
movimento: boxe_g1.npz · 1200 quadros a 120 Hz · 10.0 s · modo portico passos de física: 4995 (2 ms cada) erro médio de junta: 0.3° · pior junta: right_shoulder_roll_joint (1.9°) queda: não saída: 0
Average error of 0.3°: with the body supported, the PD handles it. The right shoulder, which moves the most in boxing, is the worst.
movimento: boxe_g1.npz · 1200 quadros a 120 Hz · 10.0 s · modo livre passos de física: 839 (2 ms cada) erro médio de junta: 0.5° · pior junta: right_shoulder_roll_joint (2.8°) queda: sim, em t = 1.68 s saída: 0
The same replay, unsupported: it falls at 1.68 s with the joint error still small. Following the angles is not balancing. That is what the tracking policy learns.
04What is used at scale (shown)
Command from the mjlab README (commit 8bb2bc9, 10/18/2026). It requires an NVIDIA GPU and a WandB account to log the motions; it was not run here and is not a prerequisite for the course.
Shown · mjlab, motion tracking on the G1 (not run here)
# github.com/mujocolab/mjlab · README (commit 8bb2bc9, 18/10/2026) uv run train Mjlab-Tracking-Flat-Unitree-G1 --registry-name your-org/motions/motion-name --env.scene.num-envs 4096 uv run play Mjlab-Tracking-Flat-Unitree-G1 --wandb-run-path your-org/mjlab/run-id
05Practice
Measure with and without the gantry.
E2 · Without gantry
5 minRun with --modo livre and note the moment of the fall.
Done when: python3 kit/reproduzir.py ~/fth/movimentos/boxe_g1.npz --modo livre | tail -1 → queda: sim, em t = 1.68 s.
E3 · The second excerpt
5 minIf you did E3 from lesson 3, replay boxe_trecho2.npz with the gantry and compare the average error and the worst joint.
Done when: python3 kit/reproduzir.py ~/fth/movimentos/boxe_trecho2.npz | grep erro → erro médio de junta: 1.5°.
06Lesson quiz
The explanation appears after your answer.
Question 1 of 3
Without the gantry, the joint error is small and the G1 still falls. Why?skill
Correct
The replay follows the angles of a human, not the forces that keep the G1 standing. The tracking policy learns to deviate a little from the reference so it does not fall.Not that one
The replay follows the angles of a human, not the forces that keep the G1 standing. The tracking policy learns to deviate a little from the reference so it does not fall.Review: Replay with the pelvis on the gantryQuestion 2 of 3
Where does BeyondMimic tracking live today, according to the README of the original repository?
Correct
whole_body_tracking (which pins Isaac Lab 2.1.0) was retired; the README points to the mjlab tracking task (10/15/2026). GMR only does the retargeting.Not that one
whole_body_tracking (which pins Isaac Lab 2.1.0) was retired; the README points to the mjlab tracking task (10/15/2026). GMR only does the retargeting.Review: Who solves motion tracking in 2026Question 3 of 3
What does the kit's virtual gantry do?
Correct
It is the simulated equivalent of the suspended robot: the body is held up, and you measure only whether the joints follow along. In FTH course 4, the real gantry shows up in the first-test procedure.Not that one
It is the simulated equivalent of the suspended robot: the body is held up, and you measure only whether the joints follow along. In FTH course 4, the real gantry shows up in the first-test procedure.Review: From reference to standing robotQuiz complete
You got {a} of {n} right. If you get them all right, the lesson is marked as complete.Choose an option. The explanation appears after you try.
07In three sentences
ReplayFollowing the angles works with the body supported and fails when it is free.
PolicyThe tracking policy looks at the state and the future reference and takes care of balance.
Where to trainmjlab (with the BeyondMimic pipeline) or a ready-made SONIC, on a GPU.