E1 · Default script
5 minRun teleop with the kit's script and 80 ms.
Done when: python3 kit/teleop_sim.py --roteiro kit/passos.json | tail -1 → alvos atingidos: 5/6.
Lesson 5 · 15 min
Whole-body teleoperation
Teleoperation is the fastest way for a humanoid to do a new task and to collect demonstrations for a VLA. The operator moves their arms in a VR headset and the robot repeats them; delay is what ruins it. In this lesson you teleoperate both of the G1's arms with a script and measure the latency at which everything falls apart.
Start the lesson
In this lesson you will
- 1The path of a gesture: VR, network, whole-body control, joint
- 2Latency × reference horizon: don't confuse it with SONIC's 80 ms
- 3Arm reach: the target that no latency can save
- 4Teleop as data collection for VLA (TWIST2, GR00T-WBC)
Goal: teleoperate both of the G1's arms through a script of targets with latency and measure how many targets the hand reaches.
01The path of a gesture
Click each block.
Click each block of the diagram to see what it does.
Teleop is a closed loop with a human in it: a big delay makes the operator correct what has already passed.
02Three things the 2026 research makes clear
Click each one.
1. 80 ms
SONIC has a "low latency" checkpoint (06/16/2026) that looks 4 future frames ahead at 20 ms: about 80 ms of reference horizon. The default checkpoint looks 10 frames ahead (about 200 ms).
The README itself warns: these numbers are not the total teleoperation latency, which includes sensor, network, preprocessing and inference.
2. VR ready
The GR00T-WholeBodyControl repository ships VR teleop with PICO, CloudXR inside the process via isaacteleop[cloudxr] (06/16/2026) and IsaacTeleop 1.4 with Jetson AGX Orin (09/14/2026).
On 05/07/2026 it gained the end-to-end VLA flow on the G1: collect with teleop, fine-tune GR00T N1.7 and deploy with SONIC.
3. One policy
HOVER (ICRA 2025, reported) distilled several command modes (VR, mocap, exoskeleton, joystick) into one policy of about 1.5M parameters.
SONIC uses a single token space for VR, human video and VLA: the same controller obeys any of them (lesson 7).
03Teleoperate both arms from a script
The kit/passos.json script plays the operator's role: six controller offsets, alternating hands. The sixth, on purpose, asks for 60 cm forward.
Terminal · teleop with 80 ms
F=${FTH:-$HOME/fth}; cd $F/corpo-inteiro && . $F/.venv/bin/activate
python3 kit/teleop_sim.py --roteiro kit/passos.json; echo "saída: $?"File · kit/passos.json
{
"descricao": "Roteiro de teleoperação: deslocamentos do controle de cada mão (em metros, a partir da posição inicial da mão), como um operador de VR moveria. Cada alvo vale até 'janela' segundos depois de enviado.",
"tolerancia_cm": 3,
"alvos": [
{"t": 0.2, "mao": "direita", "dx": 0.20, "dy": 0.00, "dz": 0.15, "janela": 0.6},
{"t": 0.9, "mao": "esquerda", "dx": 0.20, "dy": 0.00, "dz": 0.15, "janela": 0.6},
{"t": 1.6, "mao": "direita", "dx": 0.25, "dy": 0.10, "dz": 0.30, "janela": 0.6},
{"t": 2.3, "mao": "esquerda", "dx": 0.25, "dy": -0.10, "dz": 0.30, "janela": 0.6},
{"t": 3.0, "mao": "direita", "dx": 0.15, "dy": -0.05, "dz": 0.05, "janela": 0.6},
{"t": 3.7, "mao": "direita", "dx": 0.60, "dy": 0.00, "dz": 0.20, "janela": 0.6}
]
}Example from the repository
latência do operador ao robô: 80 ms · tolerância: 3 cm · controle a 50 Hz alvo 1 · t=0.2 s · mão direita · atingido em 162 ms alvo 2 · t=0.9 s · mão esquerda · atingido em 160 ms alvo 3 · t=1.6 s · mão direita · atingido em 174 ms alvo 4 · t=2.3 s · mão esquerda · atingido em 154 ms alvo 5 · t=3.0 s · mão direita · atingido em 166 ms alvo 6 · t=3.7 s · mão direita · não atingido (chegou a 21.7 cm) robô de pé: sim alvos atingidos: 5/6 saída: 0
Each target takes about 80 ms of latency + 80 ms of motion. The 6th is not reached at any latency: it is out of the arm's reach.
ERRO no roteiro: mão 'esquerdo' (use direita ou esquerda) saída: 1
The script accepts only "direita" and "esquerda". A typo stops before touching the robot, not in the middle of a session.
alvos atingidos: 6/6
The 6th target with dx 0.25 instead of 0.60: all reached.
04Where the delay breaks things
The same script with five latencies. Each target's window is 0.6 s.
Terminal · sweep the latency
F=${FTH:-$HOME/fth}; cd $F/corpo-inteiro && . $F/.venv/bin/activate
for ms in 0 80 300 450 550; do
echo "$ms ms → $(python3 kit/teleop_sim.py --roteiro kit/passos.json --latencia-ms $ms | tail -1)"
doneExample from the repository
0 ms → alvos atingidos: 5/6 80 ms → alvos atingidos: 5/6 300 ms → alvos atingidos: 5/6 450 ms → alvos atingidos: 4/6 550 ms → alvos atingidos: 0/6
Up to 300 ms there is window to spare. At 450 ms one target already misses the window; at 550 ms none is reached in time. With a real operator, the effect shows up sooner: they start correcting for the delay.
05Practice
Script, latency and reach.
E2 · 6 of 6
10 minCopy the script to ~/fth/movimentos/passos-meu.json and bring the 6th target within reach (dx less than 0.3).
Done when: python3 kit/teleop_sim.py --roteiro ~/fth/movimentos/passos-meu.json | tail -1 → alvos atingidos: 6/6.
E3 · The latency that zeroes everything
5 minRun the latency sweep and say, in one sentence, why 550 ms zeroes everything with a 0.6 s window.
Done when: python3 kit/teleop_sim.py --roteiro kit/passos.json --latencia-ms 550 | tail -1 → alvos atingidos: 0/6.
06Lesson quiz
The explanation appears after your answer.
Question 1 of 3
SONIC's low-latency checkpoint has 4 frames at 20 ms. What are those 80 ms?skill
Correct
This is how much of the future reference the policy sees. The README says it is not the end-to-end latency, which adds up sensor, network, preprocessing, and inference.Not that one
This is how much of the future reference the policy sees. The README says it is not the end-to-end latency, which adds up sensor, network, preprocessing, and inference.Review: Three things the 2026 research makes clearQuestion 2 of 3
Target 6 is not reached even with zero latency. What does that tell you?
Correct
60 cm in front of the hand is beyond the G1's arm length. The hand gets to 21.7 cm and stops: the IK does the best it can within the limits.Not that one
60 cm in front of the hand is beyond the G1's arm length. The hand gets to 21.7 cm and stops: the IK does the best it can within the limits.Review: Teleoperate both arms from a scriptQuestion 3 of 3
What is whole-body teleoperation good for besides operating the robot in real time?
Correct
Teleop is the source of task data: TWIST2 reported 100 demonstrations in 15 min, and GR00T-WBC has the flow collect → fine-tune N1.7 → deploy with SONIC.Not that one
Teleop is the source of task data: TWIST2 reported 100 demonstrations in 15 min, and GR00T-WBC has the flow collect → fine-tune N1.7 → deploy with SONIC.Review: Three things the 2026 research makes clearQuiz complete
You got {a} of {n} right. If you get them all right, the lesson is marked as complete.Choose an option. The explanation appears after you try.
07In three sentences
PathGesture, network, whole-body control, joint: each step adds delay.
LatencyReference horizon is not end-to-end latency.
DataTeleop is how you collect the demonstrations that train the VLA.