E1 · Error before and after
3 minRun the correction with the log from lesson 3.
Done when: python3 kit/delta.py ~/fth/log.npz; echo "saída: $?" → erro de rastreio sim × real: antes 8.16° → depois 3.82° and saída: 0.
Lesson 5 · 15 min
Fix it with robot data: delta action and friends
Sometimes the gap doesn't fit in two parameters. The idea behind ASAP (RSS 2025) is to learn the missing action: a model that, added to the command inside the simulator, makes it repeat what the real robot did. In this lesson you fit a minimal version of that idea with the bench log and measure the simulator's error before and after.
Start the lesson
In this lesson you will
- 1The four ASAP stages: pre-training, collection, delta model, fine-tuning
- 2A linear toy of the idea: 8 coefficients by least squares
- 3Sim × real error from 8.16° to 3.82°
- 4When to use sysid, when to use delta action
Goal: fix the simulator with robot data using a delta action model and measure the sim × real tracking error before and after.
01The ASAP cycle
Click each block. The four stages are in the README of LeCAR-Lab/ASAP.
Click each block in the diagram to see what it does.
ASAP showed agile skills on the G1 with this cycle. The kit only does stages 2 and 3, in miniature.
02The kit's toy (and what it is not)
delta.py, with the log.npz from lesson 3. Click each step.
1. The model
Per leg joint: δ = a·(target − q) + b·dq, recalculated every 20 ms (the policy step) and added to the target inside the simulator.
Four joints, two coefficients each.
2. The fit
mujoco.minimize.least_squares looks for the 8 numbers that make the simulator, replaying the log's command with the correction, reach the positions the robot measured.
Comparing the whole trajectory, rather than one step at a time, keeps the hidden latency from confusing the fit.
3. The result
A negative a pulls the target toward the current position: the simulator is imitating a softer, slower motor. Together with the kp of 0.6 and the 20 ms latency (delay), that makes sense.
4. What it is not
This is not ASAP: there, the delta model is a network with the full state, trained by RL on GPU, and the policy is tuned afterward. Here it is 8 coefficients and no retraining.
It shows the idea working on CPU, in minutes.
03Fit the correction
It uses the same log.npz from lesson 3. It takes 1 to 2 minutes on CPU (each evaluation repeats the 4 s of the bench test).
Terminal · delta action (linear toy)
F=${FTH:-$HOME/fth}; cd $F/sim2real && . $F/.venv/bin/activate
python3 kit/delta.py $F/log.npz; echo "saída: $?"Example from the repository
log: ~/fth/log.npz · correção recalculada a cada 20 ms · δ = a·(alvo − q) + b·dq por junta left_hip_pitch a = -0.66 b = -0.013 left_hip_roll a = -0.48 b = +0.006 left_knee a = -0.66 b = +0.011 left_ankle_pitch a = -0.71 b = +0.031 erro de rastreio sim × real: antes 8.16° → depois 3.82° (RMS das posições) saída: 0
Before, the simulator misses the leg positions by 8.16° (RMS). With the correction, 3.82°. It exits 0 when the error drops.
The first version of this script fit δ segment by segment (20 ms), linearizing the simulator in each segment, and the error got worse instead of dropping. The 20 ms latency (delay) makes an isolated segment look like an even softer motor, and the correction overshoots. Fitting against the whole trajectory fixed it: always check the error after the correction, never just the fit.
04Practice
Make the simulator imitate the robot.
E2 · Read the coefficients
5 minCompare the a values of the four joints with the kp identified in lesson 3 and write, in one line, why they all came out negative.
Done when: one line citing the four a values (between −0.48 and −0.71) and the word "soft" or "latency (delay)".
E3 · Sysid or delta?
5 minFor each case, write which one you would use: (1) the new battery made the robot 2 kg heavier; (2) the knee transmission has backlash that changes with temperature.
Done when: (1) sysid/measure the mass; (2) delta action or actuator network, with the reason in one sentence each.
05Lesson quiz
The explanation appears after your answer.
Question 1 of 3
In ASAP, what is the delta action model for?skill
Correct
It corrects the simulator, not the robot. With the simulator corrected, the policy is tuned again and goes back to the robot.Not this one
It corrects the simulator, not the robot. With the simulator corrected, the policy is tuned again and goes back to the robot.Review: The ASAP cycleQuestion 2 of 3
delta.py fits 8 coefficients. What sets it apart from the real ASAP?
Correct
The kit shows the idea in small scale and on CPU. Saying what it is not is part of the method.Not this one
The kit shows the idea in small scale and on CPU. Saying what it is not is part of the method.Review: The kit's toy (and what it is not)Question 3 of 3
Fitting segment by segment made the error worse; against the whole trajectory, it got better. Why?
Correct
With 20 ms of latency (delay), the response of one segment depends on the command from the previous segment. Looking at the whole lets the fit see that.Not this one
With 20 ms of latency (delay), the response of one segment depends on the command from the previous segment. Looking at the whole lets the fit see that.Review: Fit the correctionQuiz complete
You got {a} of {n} right. Get them all right and the lesson is marked as done.Pick one answer. The explanation appears after you try.
06In three sentences
IdeaDelta action learns the action that is missing for the simulator to imitate the robot.
MeasurementEight coefficients take the sim × real error from 8.16° to 3.82°.
ChoiceKnown parameter: sysid. Effect without a clear parameter: delta action.