Illustrative diagram: images (vision) + a language instruction go into a single end-to-end model, which outputs action tokens for the robot to execute.
🧩 What is a VLA model
VLA stands for Vision-Language-Action: a single model that receives camera images plus an instruction in natural language and directly generates the robot’s motor actions — the so-called action tokens. Instead of three separate systems (perception, planning, and control) stitched together by hand, VLA combines all three in one model end-to-end.
🧠 The robotics foundation model
Just as an LLM is a foundation model for text, the VLA aims to be a foundation model for robotics: trained at scale, it generalizes to tasks it has never seen (zero-shot) and transfers knowledge across distinct tasks.
- •End-to-end: a single network goes from image + text to motor command, with no manual steps in between.
- •Action tokens: the motor action is treated as a sequence of tokens, just like a language model predicting words.
- •Zero-shot generalization: ability to execute instructions for objects and scenes outside the training set.
📊 The conceptual leap
- 3 systems → 1 model: perception, reasoning, and control are no longer separate modules.
- Images + text → action tokens: the same architecture that generates words starts generating movements.
- Transfer between tasks: learning to "pick" helps you learn to "stack," reusing the representation.
💡 Practical Tip
Think of the VLA as "the robot's ChatGPT": you describe the task in English, it sees the scene through the camera, and it acts. What changes in your work is that the interface becomes language + examples, not trajectory code.
Key concepts
🤖 Reference Models
The VLA field exploded between 2023 and 2025. Knowing the leading models helps you understand their maturity and licensing—which vary widely across open-source is closed, and that determines what you can actually use.
Google RT-2 / RT-X (2023)
From the web to the robot
Google’s first large-scale VLAs: they transfer knowledge learned on the web to robot control, showing that internet vision-language data improves motor actions.
OpenVLA (2024)
~7B parameters, open-source
Open model with around 7 billion parameters. Because it is open-source, it became a foundation for the community to fine-tune locally — democratizing access to VLAs.
π0 — Physical Intelligence (2024)
Flow matching
The π0 (pi-zero) model uses flow matching to generate continuous, fluid actions instead of discrete tokens, improving dexterity in manual tasks.
Figure Helix (2025)
Real-time humanoid
Figure's VLA running on a humanoid with real-time control, showing that the approach is already moving from the lab to commercial hardware.
Google Gemini Robotics (2025)
Reasoning + action
It combines the reasoning of a multimodal foundation model with robotic action, bringing “thinking” and “doing” closer together in one system.
📊 Maturity and license
- Open: OpenVLA (~7B) enables local fine-tuning and model auditing.
- Closed / commercial: RT-2/RT-X, Figure Helix, and Gemini Robotics lag behind manufacturers’ platforms.
- Technical approach: discrete tokens (RT-2) to continuous flow matching (π0) — affects dexterity and fluidity.
Key concepts
🗣️ How the Worker Operates
With VLA, operating a robot is no longer about programming trajectories. The worker gives a natural language instruction and, when the robot makes a mistake, corrects it by demonstration — the “show it once” approach. The new skill is writing a good task prompt is to curate examples, not write traditional code.
⌨️ Illustrative example: prompt → action
Illustrative recreation of the flow, not the literal output of a real model.
✓ What the operator DOES now
- ✓Write the task prompt in natural language
- ✓Demonstrates the task through teleoperation (“show once”)
- ✓Makes on-the-spot corrections when the robot deviates
- ✓Curate which examples become behavioral models
✗ What the operator STOPS doing
- ✗Program a trajectory point by point by hand
- ✗Rewrite low-level logic for each new part
- ✗Relying only on an engineer to change behavior
- ✗Treating the robot as a fixed-revenue machine
💡 Practical Tip
Language grounding is key: the robot needs to connect the word "red" to the right pixel. When it gets it wrong, don't rewrite code—redo the demonstration or adjust the prompt. You teach; you don't program.
Key concepts
🎓 Train and Adjust
A VLA is built from massive datasets — like the Open X-Embodiment — and is then refined by fine-tuning with local demonstrations of its operation. The human stays at the center of the loop (human-in-the-loop): collecting, validating, and correcting. The bottleneck isn't the model, it's the quality data.
📚 From massive dataset to local fine-tuning
The base model learns broad patterns from datasets such as Open X-Embodiment; fine-tuning with demonstrations from your line specializes it for real parts and stations. Imitation learning means the robot copies what a human demonstrated.
- •Imitation learning: the model learns by imitating human demonstrations.
- •Data curation: choose and clean which examples go into training.
- •Edge case coverage: ensure examples of the rare situations that break the robot.
📊 The bottleneck is the data
- Open X-Embodiment: collaborative dataset that brings together demonstrations from many different robots.
- Local fine-tuning: a few good demonstrations are worth more than many noisy ones.
- Human-in-the-loop: the human collects and validates—and is responsible for data quality.
🧑🏫 Connection to the "AI Training Associate" role
A dedicated role emerges: the AI Training Associate, responsible for collecting demonstrations, curating data, and validating the robot’s behavior. This is the work of feeding and tuning the VLA day to day—exactly where the quality-data bottleneck lies.
Key concepts
⚠️ Limits and risks
VLA is powerful, but fallible. Outside the training distribution (out-of-distribution) the model makes a mistake, and worse: it may make a mistake confidently—the call motor hallucination, in which it performs an incorrect action without hesitation. That’s why safety layers and human supervision remain mandatory.
🚫 WARNING: VLA does not replace the safety layers
A confident VLA is not a correct VLA. When the scene falls outside what it saw in training, it may generate a dangerous action with high confidence—and give no warning that it is wrong. Physical safeguards and the standard still apply:
- •Force Limits and hardware-based emergency stops (e-stops), independent of the model.
- •Human Supervision active: no one hands over 100% of the operation to the model.
- •ISO 10218 and human-in-the-loop remain mandatory, VLA or not.
✓ Responsible VLA deployment
- ✓Force limits and e-stop on a layer independent of the model
- ✓Human supervision with the authority to interrupt
- ✓Compliance with ISO 10218 maintained
- ✓Monitor out-of-distribution cases and stop when uncertain
✗ Mistakes that lead to accidents
- ✗Trusting the model's confidence as proof of correctness
- ✗Operate outside the distribution without safeguards
- ✗Remove human supervision “because the VLA is good”
- ✗Treating the safety standard as optional
💡 Practical Tip
The right question before trusting a VLA action: “Does this scene look like what it was trained on?” If not, it's out of distribution—ask for human confirmation. The physical safety layer exists precisely for when the model makes a confident mistake.
Key concepts
🔮 What Changes at Work
The underlying shift: the skill moves from encode motion for teach and supervise behavior. A new role emerges—the "robot instructor" or Physical AI operator — and they're the ones who define the reskilling curriculum for this learning path.
✗ The skill that loses value
- ✗Code trajectories and movement logic point by point
- ✗Reprogram the machine every time the part changes
- ✗Think of the robot as a fixed-revenue executor
✓ The skill that gains value
- ✓Teach behavior through language and demonstration
- ✓Supervise, validate, and correct what the VLA does
- ✓Curate data and cover edge cases — the role of Physical AI
🧑🔧 The new role: robot instructor / Physical AI
Operating physical AI is an emerging profession: less “robot programmer,” more “robot teacher and supervisor.” The reskilling curriculum is organized around these competencies.
- •Robot instructor: teaches tasks through prompts and demonstrations.
- •Physical AI operator: safely supervises the VLA in production.
- •Reskilling curriculum: prompting, demonstration, data curation, and supervision.
💡 Practical Tip
Those who “program the robot” today have the shortest path to becoming a robot instructor: they already understand the task and the factory floor. What they still need to learn is task prompting, demonstration, and data curation—exactly what this learning path provides.
Key concepts
🏁 Module Summary
Next Module: 2.8
We continue the Action Path with the next practical step after mastering VLA models.