π§ Dual-System Architecture (S1/S2)
Kahneman applied to robots: fast reflexes (S1) and slow planning (S2) working in sync. The frequency separation that makes a robot that thinks possible e age at the same time.
Detailed content
𧩠Kahneman and the Thinking Fast and Slow framework
Daniel Kahneman proposed that human thinking operates in two modes: System 1 (fast, intuitive, automatic) and System 2 (slow, deliberate, analytical). In robotics, this framework inspires architectures where one reactive module controls movements in real time while another plans at a high level.
β When S1/S2 works well
- Clear separation of responsibilities β planning and control with distinct latencies.
- Asynchronous buffer β S1 never waits for S2; it continues with the last subgoal.
- Status feedback β S1 reports progress; S2 replans when needed.
- Graceful degradation β if S2 fails, S1 maintains safe behavior.
β Common pitfalls
- Synchronous pipeline β making S1 wait for S2 at every frame stalls motor control.
- Poorly sized frequencies β S2 at 1 Hz with S1 at 200 Hz causes subgoal drift.
- No watchdog β S2 stuck without a fallback leaves the robot in an undefined state.
- High-coupling interface β changes in S2 break S1 and vice versa.
Key concepts
Make optimal decisions within limited computational resources.
Time between perception and action β critical for closed-loop stability.
Faster decisions are less accurate; the dual system balances both extremes.
π System 2: VLM as a planner at 7β9 Hz
Robotic System 2 uses a Vision-Language Model (VLM) that processes images and natural language to generate high-level action plans. It typically runs at 7β9 Hz because it is computationally expensive, but provides rich semantic understanding of the environment and objectives.
π― Why 7β9 Hz is sufficient for planning
High-level planning doesnβt need speed: the task "pick up the cup" changes every second, not every millisecond. The VLM has time to reason about context, language, and visual semantics before issuing a new subgoal.
- Task decomposition β breaks natural-language instructions into executable subgoals.
- Subgoal generation β emits embeddings or tokens that S1 interprets as behavioral targets.
- Adaptive replanning β detects failures via feedback from S1 and generates a new plan.
π‘ Practical tip
In practice, the S2 VLM is almost always a pretrained model (Gemini, GPT-4V, InternVL) with minimal fine-tuning for the robotics domain. You rarely train S2 from scratch β the computational cost is prohibitive. The value lies in efficient fine-tuning with demonstration data.
β‘ System 1: visuomotor policy at 200 Hz
System 1 is a lightweight neural network β typically a diffusion policy or MLP β which converts visual observations into motor commands at 200+ Hz. It does not "understand" the overall goal; it only executes the motor behavior corresponding to the subgoal received from S2.
β±οΈ Why 200 Hz Matters
β Insufficient β shaky movements, unable to react to disturbances in real time.
β οΈ Marginal β works for slow tasks; unsuitable for fine manipulation.
β Suitable β natural compliance, reaction to disturbances, dexterous manipulation.
β Ideal β impedance control; requires dedicated hardware (FPGA/ASIC).
Key S1 concepts
Predicts action sequences instead of individual actions β reduces jitter and improves smoothness.
Models the action distribution through iterative denoising; generates smooth, multimodal trajectories.
Continuously adjusts actions based on sensory observations β essential for robustness.
Total delay from camera β inference β motor command; it must be <5ms for 200 Hz.
π€ GR00T, Helix, and Ο0: Real-World Implementations
In 2025β2026, three systems became benchmarks for how the dual-system architecture is implemented in industry. Each solves the problem with a different approachβfrom modular pipelines to unified models with a dual head.
π Design trade-offs
- Modular (GR00T/Helix) β easier to update S2 without reprocessing S1; higher communication latency.
- Unified (Ο0) β shared representations; more efficient; more complex updates.
- External model (Helix/Gemini) β takes advantage of cutting-edge LLMs without training costs; depends on a stable API.
π₯οΈ Embedded GPU: why no cloud
Autonomous robots need onboard GPUs β such as Jetson Thor or Orin β to run inference locally. Network latency (50β200 ms) makes the cloud unviable for real-time motor control, where every millisecond determines stability and safety.
βοΈ Jetson Thor / Orin
NVIDIA SoC with integrated Ada/Ampere GPU. Thor delivers 2000 sparse TOPS for robotic model inference; Orin, 275 TOPS. Designed for a robotβs thermal envelope (~20β30W).
π INT8/INT4 quantization
Reduces FP32 weights to 8 or 4 bits β 4β8x less memory, 2β4x more throughput. Essential for running 7B-parameter VLMs in limited onboard VRAM without critical accuracy loss.
π‘οΈ Thermal throttling
When the chip heats up, it reduces its clock speed to protect the hardware. On robots, this can degrade inference frequency at critical moments. Proper thermal design is part of the S1/S2 pipeline.
βοΈ Selective Offloading
Noncritical tasks (logging, long-term replanning, map updates) can use the cloud with tolerable latency. S1 is always local; S2 can have hybrid components.
π‘ Practical sizing rule
For S1 at 200 Hz: the model must take no more than 4 ms per inference. For S2 at 9 Hz: up to 100 ms per inference. This defines the maximum model size without quantizationβgenerally <100M params for S1 and <7B for S2 on Orin.
π S1βS2 orchestration: frequencies and interfaces
Orchestration defines how S1 and S2 communicate: S2 sends subgoals (embeddings or tokens) to S1, which executes and reports status. The challenge is synchronizing a slow pipeline (7 Hz) with a fast one (200 Hz) without bottlenecks β using asynchronous buffers and message prioritization.
β Recommended patterns
- Double buffering β S1 reads from the active buffer while S2 writes to the inactive buffer; no locks.
- Asynchronous message passing β S2 publishes subgoals to a topic; S1 consumes one in the next cycle.
- Watchdog timer β if S2 doesn't update for N cycles, S1 enters safe mode.
- Graceful degradation β S1 keeps the last valid subgoal until S2 recovers.
β Anti-patterns
- Synchronous blocking β S1 waits for S2 every cycle β 100 ms latency instead of 5 ms.
- High-cost polling β checking S2 every frame consumes CPU that should be used for inference.
- No failure handling β S2 stuck β S1 has no subgoal β the robot freezes or moves to the default position.
- Stale subgoal not detected β S1 executes an outdated subgoal without knowing the environment has changed.
π S1βS2 communication pipeline
VLM receives frame + instruction β infers plan β serializes subgoal into an embedding.
Subgoal written to the inactive buffer; atomic flag indicates availability to S1.
Policy reads the active subgoal β infers an action at 200 Hz β sends torques β reports status.
Receives S1 status β detects failure or progress β decides whether to replan.
β Module Summary
Next module
3.2 β World Models and Predictive Simulation