Hardware: Arms, Sensors, Edge Computing

Detailed content

🔹 Manipulator arms: SO-101, Franka, UR, xArm

Arms differ along three axes: DOF (degrees of freedom), payload e repeatability. O SO-101 is the entry point (~$100, Feetech servos); Franka Panda is the research standard with 7 DOF and torque/force control; UR e xArm are industrial cobots.

📊 Quick comparison

SO-101 — ~$100, 6 DOF, ideal for learning and collecting demos.
Franka Panda — 7 DOF, force control, an academic favorite.
UR5e / UR10e — industrial cobot, safety certification.
xArm 7 — good research value, medium payload.

# controlar um Franka via libfranka / franka_ros2 (controle de torque)
ros2 launch franka_bringup franka.launch.py robot_ip:=172.16.0.2
# enviar setpoints de junta a partir da policy VLA -> /joint_trajectory
# repetibilidade típica: Franka ±0.1mm · SO-101 ~mm (servos de hobby)

⚡ Practical tip

Control by torque/admittance (Franka) enables safe contact; position-only arms require great care during insertion. To get started with VLA, the SO-101 covers 90% of the learning at 1% of the cost.

DOF

Degrees of freedom.

Payload

Maximum payload.

Repeatability

Return accuracy.

Force control

Safe contact.

🔹 Grippers and hands: parallel, suction, multi-finger, tactile

O end-effector defines what the robot can grasp. Gripper parallel is simple and robust; suction dominates bin-picking from flat surfaces; hands multi-finger (Allegro, Shadow) provide dexterity at the cost of complexity. Tactile sensors such as GelSight provide contact feedback that the camera can’t see.

📊 Choosing the end effector by task

Pick-and-place — a parallel gripper gets the job done.
Bin picking for boxes/plates — suction.
Fine manipulation / in-hand — multi-finger + tactile.
Insertion (peg-in-hole) — force/admittance + GelSight.

✓ Do

✓Start with a parallel gripper: it covers most tasks.
✓Add tactile sensing for insertion/fine manipulation and in-hand manipulation.
✓Match the end effector to the task, not the other way around.

✗ Avoid

✗A 20-DOF hand for a simple pick-and-place task.
✗Suction on porous or irregular objects.
✗Ignore the gripper state in VLA observations.

⚡ Practical tip

Tactile sensing is the frontier: policies vision-tactile (e.g., integrating GelSight into the VLA) solve insertion and manipulation of translucent/slippery objects that the camera alone cannot see. But add touch only when the task calls for it—an extra sensor is one more axis for the model to learn.

Parallel

Simple and robust.

Suction

Planar bin picking.

Multi-finger

Dexterity, in-hand.

GelSight

High-resolution tactile sensing.

🔹 Sensors: RGB-D, wrist cam, IMU, encoders

Vision is the VLA's dominant input. RealSense (D435/D455) e ZED provide affordable RGB-D; the wrist cam (on the wrist) resolves occlusion and improves fine manipulation. Encoders joint close the proprioception and the IMU helps with mobile bases/humanoids.

# RealSense via SDK / ROS2
ros2 launch realsense2_camera rs_launch.py \
  enable_depth:=true align_depth.enable:=true rgb_camera.color_profile:=640x480x30

# múltiplas câmeras: top + wrist viram chaves no LeRobotDataset
# observation.images.top / observation.images.wrist

📊 Role of each sensor

Top camera — global view of the scene, task context.
Wrist cam — local detail, anti-occlusion at the gripper.
Encoders — joint state (part of the observation.state).
F/T sensor — wrist force for controlled contact.

RGB-D

Color + depth.

Wrist cam

Anti-occlusion.

Encoder

Joint state.

IMU

Acceleration/guidance.

🔹 Edge computing: Jetson Orin/Thor, throughput vs. power consumption

Run the VLA on-board eliminates network latency. The line Jetson dominates: Orin (AGX/NX) delivers tens to hundreds of TOPS within a 15–60 W thermal envelope; Thor (Blackwell, 2025) raises the bar for edge robotics foundation models. The key is matching quantization to the power envelope.

Choose the module

Orin Nano/NX for small policies; AGX Orin/Thor for 3-7B VLAs at the edge.

Optimize

TensorRT, INT8/FP8, kernel fusion; cut latency without losing much success rate.

Manage thermal/power

Power mode (nvpmodel), jetson_clocks and heat dissipation determine sustained throughput.

# fixar modo de potência máximo e clocks no Jetson
sudo nvpmodel -m 0          # MAXN
sudo jetson_clocks
tegrastats                  # monitorar GPU/CPU/EMC e temperatura

# compilar a policy com TensorRT (INT8) para inferência on-board
trtexec --onnx=policy.onnx --int8 --saveEngine=policy.engine

Jetson Orin

Edge, 15–60 W.

Jetson Thor

Blackwell for FMs.

TensorRT

INT8/FP8 at the edge.

Thermal envelope

Sustained throughput.

🔹 Teleoperation: leader-follower, VR, gloves, ergonomics

All imitation starts with teleoperation. Leader-follower (ALOHA, SO-101) is the most direct: one arm mirrors the other. VR e gloves capture dexterous hands and map them to the robot. The ergonomics matters: tedious data collection produces poor demos, and poor data goes straight into the model.

📊 Teleoperation modalities

Leader-follower — inexpensive, intuitive, matches the robot’s action space.
VR (Quest/Vision Pro) — 6-DOF hand capture, good for humanoids.
Gloves — multi-finger hands, dexterity data.
Kinematic mirroring — retarget human poses to the robot.

⚡ Practical tip

Make sure the action space for the teleop to be identical to what the policy commands (Δpose vs. joints). Any discrepancy there creates systematic bias that the VLA learns as noise.

Leader-follower

Direct mirroring.

6-DOF capture.

Gloves

Multi-finger dexterity.

Ergonomics

Comfort = good data.

✓ Do

✓Match the teleop action space to what the policy commands.
✓Short, comfortable sessions to maintain quality.
✓Deliberately vary the object's position (coverage).

✗ Avoid

✗Action space discrepancy (Δpose vs joints).
✗Exhaustive collection that degrades demo quality.
✗Record 200 identical episodes with no variation.

🔹 Budget and assembly: from $100 to a research platform

The cost of entry has plummeted. A complete VLA learning setup— SO-101 leader + follower + 2 cameras + Jetson — costs hundreds of dollars, compared with tens of thousands for a Franka cell. Today, the barrier is less about money and more about methodology.

📊 Budget ranges

$100–300 — SO-100/SO-101 single, USB camera, trains on a laptop/cloud.
$500–1k — leader+follower, 2 cameras, Jetson Orin Nano.
$10k+ — Bimanual ALOHA, multiple RealSense cameras, workstation.
$50k+ — Franka/UR with F/T, a complete research cell.

⚡ Practical tip

Start cheap, train in the cloud, and run inference on the edge later. Assembling the SO-101 + LeRobot is the shortest path from zero to a policy running on a real arm.

Input

SO-101 ~$100.

Intermediate

Leader+follower+Jetson.

Research

ALOHA / Franka.

Method > budget

Good data matters more.

✅ Module summary

✓

Arm by DOF/payload/repeatability — SO-101 is gaining traction; Franka is used in research.

✓

End-effector follows the task — parallel, suction, multi-finger, tactile.

✓

Sensors — RGB-D + wrist cam + encoders feed the VLA.

✓

Jetson + quantization — on-board inference matching power and throughput.

✓

Cost dropped sharply — Accessible VLA; methodology matters more than budget.

Next module

2.6 — Datasets: Open X-Embodiment, DROID, and LIBERO, the fuel that trains everything we’ve seen.

← Back to the track Next module →