🔢 Action Tokenization: Discrete vs Continuous vs FAST

The most natural representation: the head outputs floats directly for each action dimension (Δx, Δy, Δz, rotation, gripper). It’s simple, preserves full resolution, and wastes nothing on discretization. The problem, as seen in Module 1.4, is that when trained with MSE/L1 it is unimodal — collapses modes. That’s why continuous regression shines only when paired with an expressive head (diffusion/flow), rather than a simple MLP+MSE.

RT-1, RT-2, and OpenVLA discretize each dimension into uniform bins (RT-1 uses 256) and treat prediction as classification via cross-entropy. The advantage: it becomes a token problem, integrating seamlessly with the LLM’s vocabulary and capturing multimodality (categorical distribution). The drawbacks: loss of resolution (quantization) and loss of correlation between dimensions, since each one is tokenized independently.

This is where the pain that motivated FAST lives. At 50Hz, actions at consecutive timesteps are almost identical — the arm moves very little in 20ms. Tokenizing each timestep generates redundant tokens and enormous sequences: a 1-second chunk at 50Hz with 7 dimensions contains 350 tokens, most of them repeated. Long sequences make autoregressive training prohibitively expensive and degrade learning. Naive binning simply doesn't scale to high-frequency dynamic control.

# Custo de binning por timestep a 50Hz
chunk = 1s · 50Hz = 50 timesteps
dims  = 7 (xyz + rot + gripper)
tokens = 50 × 7 = 350  ← maioria redundante
# treino autoregressivo nessa sequência: lento e instável

FAST (Frequency-space Action Sequence Tokenization, Physical Intelligence, 2025) reverses the logic: instead of tokenizing in time, it transforms the trajectory into frequency domain via DCT (the same transform as JPEG), quantize the coefficients, and compress them with BPE. Since smooth signals concentrate energy in a few low frequencies, the result is few high-fidelity tokens. FAST makes autoregressive VLAs competitive with diffusion in high-frequency control—RT-2/OpenVLA-style, but dexterous.

Every tokenization lives in a Pareto frontier: fewer tokens train faster but may lose fidelity; more tokens preserve detail but cost more in training and latency. Naive binning lands in a bad spot (many tokens, average fidelity). FAST shifts the frontier — few tokens AND high fidelity — by exploiting the signal's frequency structure. The right choice depends on how compressible your signal is.

Esquema     Tokens/chunk   Fidelidade   Treino
contínuo    1 (vetor)      total        rápido*  (*precisa head expressiva)
binning     muitos         média        lento
FAST        poucos         alta         rápido

There is no universal tokenization—the choice shapes the entire action head. Slow, quasi-static task (paused pick-and-place)? Binning works. Dexterous, dynamic manipulation (folding clothes, insertions)? FAST (autoregressive) or diffusion/flow. The representation choice determines whether the head is a detokenizer, a generative action expert, or a regressor—closing the conceptual loop that began in Module 1.2.