FTH 1 · Teaching it to walk|INEMA.CLUB-PRO

FTH — Humanoid Technology Training · course 1 of 5 · 8 lessons

Teaching it to walk: locomotion by RL in simulation

The Unitree G1 walks because a learned policy sends 29 joint targets every 20 ms. In eight lessons you run a ready-made policy, tune the PD, build the environment, read the reward term by term, train PPO on CPU, read the curves, evaluate with a push, and write your own training plan.

Start with lesson 1 See the kit on GitHub
Graphite-gray industrial humanoid robot with a black visor, mid-step in a dark lab, attached to a steel gantry, with the title Teaching it to walk.
  • 8lessons
  • 128minutes of lessons
  • 24exercises with "done when"
  • 20/20episodes upright with a push (ready-made policy)

The lessons

From the ready-made policy to your training plan: PD on the joints, environment, reward, PPO, curves, evaluation, and terrain curriculum.

  1. Lesson 1 · 16 minWhy locomotion became RLThe Unitree G1 in this course walks because a neural network sends 29 joint targets every 20 ms. In this lesson you set up the bench, run the ready-made Unitree G1 gait policy in MuJoCo, watch the same robot fall without it, and see why classical control gave way to RL in massive simulation.
  2. Lesson 2 · 16 minThe bottom layer: PD control at the jointsBetween the policy and the motor sits a PD controller on each joint, with kp and kd chosen during training. In this lesson you measure a knee's response to a step with three kd values, hold the G1 upright with PD alone, and find out why the policy's gains can't keep the robot standing on their own.
  3. Lesson 3 · 16 minObservation, action, reward, and end of episodeBefore training, the problem has to become an environment: what the policy sees, what it sends, how much it earns, and when the episode ends. In this lesson you read the four parts in the "manager-based" format of Isaac Lab and mjlab, in a minimal version in plain MuJoCo, and you run one episode with three different policies.
  4. Lesson 4 · 16 minDesigning the gait rewardThe policy learns what the reward pays for, not what you meant. In this lesson you read the gait terms of the mjlab velocity task, record one rollout walking and another standing still, measure how much each term paid, and find out that, with one weight swapped, standing still becomes worth more than walking.
  5. Lesson 5 · 16 minPPO with thousands of robots in parallelThe algorithm that trains almost every humanoid gait today is PPO, running thousands of copies of the robot on the GPU. In this lesson you learn the collect, estimate, and update loop, run a ~120-line PPO with mjlab's values on CPU, with 16 robots, and see the command that does the same with 4,096.
  6. Lesson 6 · 16 minReading the training: curves, collapse, and reward hackingA reward curve going up does not mean the robot learned to walk. In this lesson you read the log of your own training and of two real 150-iteration trainings, recorded for the course: in one of them the reward goes up because the robot learned to fall sooner. The other shows the fix and the limit of the fix.
  7. Lesson 7 · 16 minEvaluating for real: speed, turning and pushA policy is only ready when it passes a test you wrote before looking at the result. In this lesson you evaluate the ready-made policy over 20 episodes with fixed seeds: speed error, turning and pushes of 1.0, 1.5 and 2.0 m/s. Then you run your checkpoint from lesson 5 through the same test.
  8. Lesson 8 · 16 minTerrain curriculum and your training planNobody trains stair walking on day one: the terrain and the requested speed grow as the policy succeeds. In this lesson you measure how far up the ladder the ready-made policy goes, read how mjlab and unitree_rl_lab level up, and write your own training plan in a file that a script checks.

Who it's for

  1. 1People who code in Python, have seen ROS 2 (or took the robot course), and want to understand how a humanoid learns to walk.
  2. 2People with no humanoid and no GPU: every practice runs on CPU, in minutes, in MuJoCo.
  3. 3People who took course 0 (the ~/fth bench): this course reuses the venv and the Menagerie.

What you walk away able to do

  1. 1Run the G1 gait policy in MuJoCo and measure what it does.
  2. 2Read each term of the gait reward in a recorded rollout and predict the effect of a weight.
  3. 3Train PPO on CPU, read the curves, and spot reward hacking.
  4. 4Evaluate with a push and a step, and write a training plan that a script checks.