🎯 Imitation Learning and Behavior Cloning

Behavior Cloning (BC) reduces control to supervised learning: given a dataset of state-action pairs demonstrated by a human, a policy π(a|s) is trained to maximize the likelihood of the observed actions. It is simple, stable, and scales — which is why almost every VLA (RT-1, OpenVLA, π0) is trained with BC. The policy needs neither a reward nor interaction with the environment during training, only demonstrations.

# Behavior Cloning — objetivo
#   maximizar  Σ log π(a_t | s_t)   sobre demos do expert
for (obs, action) in demo_loader:
    pred = policy(obs)               # estado → ação
    loss = criterion(pred, action)   # ex: cross-entropy ou flow
    loss.backward(); opt.step()

The Achilles' heel of BC: the model is trained on the state distribution of the expert, but at runtime it visits states it generated itself. A small error steers the robot into a slightly out-of-distribution state, where the policy is less reliable, makes a bigger error, and so on—the compounding error. Theoretically, the cost grows with O(εT²) over horizon T, versus the ideal O(εT).

DAgger (Dataset Aggregation) tackles distribution shift by collecting labels in the states the policy actually visits: the clone is run, and the expert annotates the correct action in those states, adding it to the dataset and retraining. This teaches recovery behaviors. The demos themselves come from teleoperation — leader-follower, VR, or gloves — whose quality and coverage set the policy ceiling.

For the same state, there are often multiple equally good actions — go around an obstacle on the left or right. Different demonstrators choose different paths. If the policy is trained with MSE, it learns to average of those actions, which may land exactly on the obstacle (mode averaging). That's why modern VLAs use heads expressive — diffusion, flow matching, or discrete tokens — capable of representing multimodal distributions.

In imitation learning, data > architecture. What matters isn't raw volume, but coverage (rare and recovery states represented), diversity (objects, positions, lighting, distractors) and consistency. Suboptimal or contradictory demos confuse the policy; clean, varied demos generalize. A small but diverse corpus often beats a large but redundant one.

The ultimate metric is success rate — fraction of attempts that complete the task. But a high number in training scenes can be misleading. An honest test measures generalization across three axes: new objects, new positions/scenes, and rephrased instructions. Always evaluate with Enough attempts and OOD (out-of-distribution) protocols, reporting confidence intervals.

# Protocolo de avaliação honesto
seen      = eval(policy, scenes_treino)      # baseline
ood_obj   = eval(policy, objetos_novos)      # generaliza objeto?
ood_pos   = eval(policy, posicoes_novas)     # generaliza cena?
ood_instr = eval(policy, instrucoes_novas)   # generaliza linguagem?
report(success_rate, n=50, ci=0.95)