E1 · Run the cycle
5 minRun the simulation to the end, stop it once in the middle, and resume.
Done when: the "Run" step shows up in falhou/ and the steps in feito/ were not redone on resume.
Lesson 7 · 16 min
Beyond imitation: RL, world models, and memory
Imitation has a ceiling: the policy only learns what the person showed it, and it doesn't know how to recover from a mistake it has never seen. In 2026 three ideas push that ceiling: learning from its own experience (RL), predicting the future in video (World Action Models), and remembering what it has already done (memory).
Start the lesson
In this lesson you will
- 1The imitate → run → fail → correct cycle
- 2RL in post-training: Recap and RL Token
- 3World Action Models: DreamZero, GR00T N2, Cosmos 3
- 4Memory for tasks longer than 10 minutes
Goal: explain RL after imitation, World Action Models, and long-term memory with 2026 examples.
01The frontier line
Scroll the page: each step lights up a part.
Scroll the page: each step below lights up its part of the diagram.
- 1/5
Imitation
What you did in lesson 5: the policy copies the actions in the dataset. It works, but it inherits the limits of the demonstrations.
- 2/5
RL afterward
After imitating, the policy runs, makes mistakes, and learns from the outcome and from human corrections. According to PI, Recap more than doubled throughput on hard tasks and cut failures by half or more.
- 3/5
World Action Model
The model predicts the video of what will happen along with the action. The bet is that predicting the future teaches physics better than predicting the action alone.
- 4/5
Memory
Remember what has already been done so you don't repeat or skip steps. PI's MEM handled tasks longer than 10 minutes, like cleaning an entire kitchen.
- 5/5
The whole frontier
Imitation gives the start; RL corrects with experience; the World Action Model predicts the future; memory holds long tasks together. The top models of 2026 combine more than one.
02The 2026 examples
Click each one. Numbers are as reported by the companies.
1. Recap
RL with corrections and advantage-conditioned policies: the policy learns what went better and what went worse in its own attempts, and from a person's corrections.
According to PI, π*0.6 made espresso from 5:30 a.m. to 11:30 p.m., folded 50 new garments, and assembled 59 boxes in a factory.
2. RL Token
The VLA learns to emit an "RL token" that summarizes what it sees. A small actor-critic trains RL in real time on that token, with just a few hours or minutes of data.
Tested on precision tasks like tightening an M3 screw with an electric screwdriver and plugging in an ethernet cable.
3. DreamZero
A 14B World Action Model that predicts video and action together and controls in a closed loop at 7 Hz. According to the paper, it generalizes more than twice as well as state-of-the-art VLAs on new tasks and environments.
GR00T N2, announced in Mar 2026 on top of this architecture, was expected for late 2026 and had no confirmed release as of Oct 2026.
4. MEM and π0.7
MEM gives the VLA short-term and long-term memory, evaluated on top of π0.6, on tasks longer than 10 minutes.
π0.7 (Apr 2026) accepts quality and speed metadata and subgoal images in the prompt, and learns through language "coaching" to use devices it hasn't seen. It is closed.
03The imitate → run → fail → correct cycle
Run the cycle. One step fails on purpose the first time: see what the correction does.
a fazer/
rodando/
feito/
falhou/
Illustrative simulation in the browser: no robot actually runs. The failure is intentional, to show where imitation stops and correction begins.
04Log the cycle
Copy the template and note each round. Without a before and after number, you can't tell which step helped.
ciclo.md
# Improvement cycle: <task> | Round | What changed | Attempts | Successes | Rate | Most common failure | |---|---|---|---|---|---| | 1 | imitation only (N demonstrations) | 20 | | | | | 2 | + human corrections (dagger) | 20 | | | | | 3 | + RL tuning | 20 | | | | Conclusion: the step that reduced failure the most was ______ because ______.
With the real arm: corrections with dagger (from the LeRobot 0.6 documentation, not run here)
lerobot-rollout \ --strategy.type=dagger \ --policy.path=<seu usuário>/minha_politica \ --robot.type=so101_follower \ --robot.port=/dev/ttyACM0 \ --teleop.type=so101_leader \ --teleop.port=/dev/ttyACM1 \ --dataset.repo_id=<seu usuário>/correcoes_dagger \ --dataset.single_task="Pegue o bloco"
Simulation: illustrative example
- Compare the rate from one round to the next with the same number of attempts.
- Change one thing per round; if you change two, you don't know which one helped.
- The dagger command needs the arm (and a Hub account to upload the dataset); it's here so you can recognize the flow.
05Practice: which step reduced the failure?
Use the simulation above and the examples.
E2 · Explain the failure
10 minWrite three lines: why imitation failed, which step handles the failure, and which 2026 example does something similar.
Done when: the three lines mention the case missing from the demonstrations, the correction or RL tuning step, and an example (Recap, RL Token, or lerobot-rollout with dagger).
E3 · Where the world model comes in
10 minIn one sentence, say how a World Action Model could help before the first run on the robot.
Done when: the sentence is about predicting the outcome in video before acting (or about using video of humans and other robots in training).
06Test what you learned
The explanation appears after your answer.
Question 1 of 4
What is the main limit of pure imitation?skill
Correct
The policy copies what it saw. A new case (the cup turned over) isn't in the data. RL and human correction address exactly that gap.Not this one
The policy copies what it saw. A new case (the cup turned over) isn't in the data. RL and human correction address exactly that gap.Review: The imitate → run → fail → correct cycleQuestion 2 of 4
What does PI's RL Token (Mar 2026) allow?
Correct
The large VLA stays as it is; a small actor-critic learns on top of the RL token with just a few hours or minutes of data.Not this one
The large VLA stays as it is; a small actor-critic learns on top of the RL token with just a few hours or minutes of data.Review: The 2026 examplesQuestion 3 of 4
What defines a World Action Model like DreamZero?
Correct
A WAM predicts the future in video and the action at the same time. DreamZero has 14B parameters and controls at 7 Hz.Not this one
A WAM predicts the future in video and the action at the same time. DreamZero has 14B parameters and controls at 7 Hz.Review: The frontier lineQuestion 4 of 4
As of Oct 2026, what is the status of GR00T N2?
Correct
NVIDIA showed a preview on 03/16/2026 and says it leads some rankings, but as of Oct 2026 no release has been found. An announcement is not a product.Not this one
NVIDIA showed a preview on 03/16/2026 and says it leads some rankings, but as of Oct 2026 no release has been found. An announcement is not a product.Review: The 2026 examplesQuiz complete
You got {a} of {n} right. Get them all right and the lesson is marked as complete.Pick an option. The explanation appears after your attempt.
07Summary
Imitate, then correctRL and human correction address what the demonstrations didn't cover.
Predict the futureWorld Action Models predict video and action together: DreamZero, GR00T N2 (announced), Cosmos 3 as a base.
RememberMemory (MEM) holds up tasks longer than 10 minutes.