๐ World Models and Predictive Simulation
Robots that imagine the future before acting: generative models as a prediction engine. From JEPA to Dreamer โ learn how agents simulate consequences in their minds before touching the real world.
Predictive simulation loop: the robot imagines multiple rollouts in the World Model and executes only the action with the highest estimated reward.
๐ง What World Models Are
World Models are internal representations an agent builds of its environment โ a kind of โmental map of the future.โ Instead of acting through trial and error in the real world, the robot mentally simulates scenarios and chooses the best trajectory before moving a single servo.
๐ฎ Core Definition
A World Model is a function f(s_t, a_t) โ s_{t+1} that, given the current state and a proposed action, predicts the next state. Chained k times, generates a rollout of k steps in the future โ the robotโs โdream.โ
The world is compressed into a low-dimensional latent space where dynamics are easier to learn than in raw pixel space.
Component that predicts the next state. It can be deterministic or stochastic (modeling uncertainty as a distribution over possible futures).
Joint Embedding Predictive Architecture (LeCun): predicts representations of future states in feature space, without reconstructing pixels โ more efficient and robust.
Simulate N candidate trajectories, evaluate each with a reward function, and execute only the action from the winning trajectory.
๐ก Why Yann LeCun is betting on this
LeCun argues that World Models are the missing piece for human-level intelligence: humans spend most of their time "imagining," not acting. JEPA is his proposal for training these models without representational collapse.
๐ฌ Generative video models: Sora, UniSim, Genie2
Models such as Sora (OpenAI), UniSim (Google), and Genie2 (DeepMind) generate realistic videos of the future conditioned on actions. In robotics, they function as visual world models: given the current frame and a proposed action, they generate the next frames showing the likely result.
โ Advantages of Video World Models
- โ Generate photorealistic simulations of unseen environments
- โ Allow policies to be trained without a traditional simulator (Isaac Sim, MuJoCo)
- โ Scale with internet video dataโtrillions of frames available
- โ Capture implicit physics: objects fall, liquids flow, shadows are cast
โ Current limitations
- โ Does not guarantee strict physical consistency (objects may teleport)
- โ Generation is too slow for real-time online planning
- โ Content errors accumulate over long rollouts (>2โ3 seconds)
- โ Difficult to condition on actions in torque/joint angle space
๐ฏ The long-term bet
If video world models achieve sufficient physical fidelity, they replace Isaac Sim and MuJoCoโtraditional simulators that require manual physics modeling. Robot training becomes as scalable as training LLMs on web text.
๐ Predicting consequences: "What if I do X?"
The robot evaluates multiple candidate actions by simulating their consequences in the world model. For each action, it generates a trajectory of future states and calculates a reward function for the predicted outcome. It then executes the action with the best estimated outcome in the real world.
Action Sampling
Sample N candidate actions from the action space. This can be done with CEM (Cross-Entropy Method), MPPI, or simply by adding Gaussian noise to the current action.
Parallel Rollout
Each candidate action is expanded k steps in the World Model in parallel on the GPU. Produces N future-state trajectories simultaneously.
Reward Prediction
A reward function evaluates each trajectory. It can be a separate neural network, distance to a goal state, or an engineering cost function.
Winnerโs Execution
The action from the trajectory with the highest cumulative reward is executed. Then the cycle restarts with the new observationโthis is Model Predictive Control (MPC).
โก Why this matters
This โlook-aheadโ mechanism is what separates reactive robots from truly intelligent robots. It makes it possible to avoid irreversible actionsโsuch as knocking over a fragile objectโby predicting the consequence before acting. The cost: planning latency.
๐ญ Planning through imagination (model-based RL)
Model-based reinforcement learning trains a policy using experiences generated by the world model instead of real interactions. Algorithms such as Dreamer and TD-MPC learn in the modelโs "dream," then transfer the behavior to the physical robotโreducing training cost and risk by orders of magnitude.
โ Established algorithms
- โDreamerV3 โ learns through imagination, works on 150+ tasks without adjustment
- โTD-MPC2 โ combines MPC with TD learning; state of the art in continuous robotics
- โMBPO โ policy optimization with short rollouts in the model (Janner et al.)
โ Common pitfalls
- โRollouts that are too long โ model errors accumulate (model bias)
- โPolicy optimizes the model, not the real world (exploitation of model error)
- โVery simple model โ does not capture the real environmentโs multimodality
๐ญ Training with synthetic world model data
In addition to online planning, world models can generate massive synthetic datasets to pretrain policies. The model generates millions of plausible trajectories with variations in objects, lighting, and configurations โ data that would be impossible to collect manually at scale.
๐ The Robotics Data Bottleneck
LLMs scaled because internet text provided trillions of tokens. Robotics doesnโt have that luxury: real trajectory data is expensive (~$10โ100/hour of collection) and scarce. World models are the proposal for closing this gap.
Generate data with randomized lighting, textures, and positionsโthe world model does this at zero marginal cost compared with moving real objects.
Fundamental trade-off: data that more faithfully reflects real-world physics is more useful, but limiting fidelity allows for greater diversity in generated scenarios.
Researchers are investigating whether robotic policy performance scales predictably with the volume of synthetic data โ analogous to the scaling law for LLMs.
World model as an augmenter: generates variations of real trajectories with state perturbations โ enriches the dataset without additional collection.
๐ The key bet
If world models become reliable physics simulators, robotics' "data bottleneck" disappears. Companies like 1X Technologies and Physical Intelligence already use this approach to scale data collection synthetically.
โ ๏ธ Limitations: When the Model Predicts Incorrectly
World models arenโt perfect: prediction errors accumulate over the time horizon (compound error), deformable objects are difficult to model, and complex multi-body interactions often diverge from reality. Blindly trusting a world model is as dangerous as having none.
โ World Model failure modes
- โCompound Error โ small errors at each step add up: after 50 steps, the predicted state may be completely unrealistic
- โDeformable objects โ fabric, liquids, soft mass: physics that current neural networks can't handle
- โNew interactions โ out-of-distribution: the model extrapolates poorly to situations it did not see during training
- โPrecise contacts โ fitting a part, tightening a screw: high-precision contact physics is critical
โ Mitigation strategies
- โShort-term horizon โ limiting rollouts to 5โ15 steps reduces error accumulation
- โUncertainty estimation โ measure model uncertainty and stop imagining when it is high
- โModel ensemble โ use disagreement between models as a proxy for uncertainty
- โReal โ Simulated Alternation โ regularly observe the real world again to correct drift
๐จ The trust paradox
A robot that trusts the world model too much executes fanciful plans that fail in the real world. A robot that distrusts it too much never acts. The balance โ knowing when imagine and when observe โ it is an open problem in uncertainty calibration.
โ Module Summary
Next module
3.3 โ Data-Driven Robotics: Open X-Embodiment and Massive Datasets