The loop that learns|INEMA.CLUB-PRO

Lesson 6 · 18 min

The risk of optimizing against your own meter

A loop that improves itself will find the shortest path to a higher score, and the shortest path isn't always improving the system: it can be memorizing the cases or tampering with the meter. In this lesson you watch both shortcuts become a keep, close them with a meter frozen outside the repo and hidden cases, and watch a real Claude change get discarded.

Start the lesson
A laboratory thermometer with a flame heating only the bulb, while a glass of ice next to it stays cold.

In this lesson you will

  1. 1Goodhart's law: a metric that becomes the target stops measuring
  2. 2Meter frozen with a hash, out of the agent's reach
  3. 3Hidden cases: the gain has to show up where the agent can't see
  4. 4"Maximize X without worsening Y" as the required form

Goal: catch the agent that improves the score without improving the system: a meter with a hash, kept outside the repo, and hidden cases the agent can't see.

01How the loop fools itself

Three shortcuts, two guards. Click each block.

The LOOP-R example lists the same pattern in business: asking for clicks teaches clickbait; asking for reply rate teaches messages that provoke "stop sending me this" (loop-r/docs/06 §4).

Detecting shortcuts in tests (conftest, skip, hidden tests) and calibrating a judge per model is covered in course 1, lessons 3, 4 and 6. Here: the loop that optimizes its own metric.

02Rehearsal: the shortcuts without a guard and with a guard

The same fake agents, first in plain experimento.sh, then with GUARDA and ESCONDIDOS.

Terminal · cost-free rehearsal
K=~/projetos/loop-r-eng-aprende-v72/kit
E=${TMPDIR:-/tmp}/ensaio-aprende/a6; rm -rf $E; mkdir -p $E
echo "## sem guarda: a métrica sobe e o sistema não melhora"
for m in decora mexe-casos; do
  bash $K/novo-sistema.sh $E/sis > /dev/null
  AGENTE="bash $K/agente-falso.sh $m" bash $K/experimento.sh $E/sis "$m"
  echo "  nos casos escondidos: $(python3 $K/medir.py $E/sis --casos $K/triagem/escondidos.jsonl)"
done
echo "## com guarda: medidor congelado fora do repo + casos que o agente não vê"
bash $K/novo-sistema.sh $E/sis > /dev/null; bash $K/experimento.sh --congelar $E/sis $E/medidor.sha256
export GUARDA=$E/medidor.sha256 ESCONDIDOS=$K/triagem/escondidos.jsonl
for m in mexe-casos decora sinonimos; do
  AGENTE="bash $K/agente-falso.sh $m" bash $K/experimento.sh $E/sis "$m"; echo "saída: $?"
done
Example from the repository
## sem guarda: a métrica sobe e o sistema não melhora
ciclo 1 · keep · 0.700 → 1.000 · melhorou
  nos casos escondidos: acerto: 0.700 (21/30)
ciclo 1 · keep · 0.700 → 1.000 · melhorou
  nos casos escondidos: acerto: 0.700 (21/30)
## com guarda: medidor congelado fora do repo + casos que o agente não vê
medidor congelado em /tmp/ensaio-aprende/a6/medidor.sha256
ciclo 1 · discard · 0.700 → 0.700 · DESCARTADO: mexeu no medidor (casos.jsonl)
saída: 1
ciclo 2 · discard · 0.700 → 1.000 · DESCARTADO: ganho só no visível (escondidos 0.700 → 0.700)
saída: 1
ciclo 3 · keep · 0.700 → 0.800 · melhorou (escondidos 0.700 → 0.800)
saída: 0

Without a guard, both shortcuts go 0.700 → 1.000 and keep, and the hidden cases stay at 21/30. With a guard: touching the cases fails on the hash, memorizing fails on the hidden cases, and only the synonyms (which really improve, 21 → 24 of 30) get through.

03The same risk in research

What the paper measures and what shortcut it reports itself.

1. Darwin Gödel Machine

An archive of agent variants; a variant stays when the benchmark passes a threshold. The paper reports SWE-bench going from 20.0% to 50.0% and Polyglot from 14.2% to 30.7%, in a sandbox with human oversight.

It's this lesson's loop at scale: the agent changes its own code and a benchmark decides.

04LOOP-R on video: measuring for real

LOOP-R track 4 (Nei Maldaner): inflated goals with no real improvement, minimum sample and guardrails.

LOOP-R, track 4: measure for real · 17 min 44 sVideo page with script →

05Practice

Rehearsal first, then your own meter.

E16 · The two shortcuts

10 min

Run the rehearsal.

Done when: without a guard, the two lines keep · 0.700 → 1.000 and the hidden cases at 21/30; with a guard, DESCARTADO: mexeu no medidor, DESCARTADO: ganho só no visível and a single saída: 0.

E17 · Freeze your meter

20 min

In your system, separate what measures (cases, measuring script, tests) from what the agent changes. Freeze what measures in a manifest outside the repo: in the repo folder, python3 $K/hashes.py congelar [manifest outside the repo] [files that measure], or bash $K/experimento.sh --congelar . [manifest] with MEDIDOR exported (lesson 2, "Adapt it to your repo").

Done when: python3 $K/hashes.py conferir [manifesto]; echo $? → 0; change a case on purpose → MUDOU: … and 1; undo it.

E18 · Your hidden cases

30 min

Set aside at least 20% of your real cases in a file outside the repo, one that never goes into the request to the agent. Run your latest keep against them. With your own measurer, ESCONDIDOS only works if the MEDIR command accepts --casos [file].

Done when: python3 $K/medir.py [repo] --casos [hidden cases] (or your MEDIR with --casos) prints the score on the hidden cases, and you noted, in results.tsv or in the ledger, whether the latest keep holds up on them.

06Lesson quiz

Question 1 of 4

The agent copied the exact text of the failed cases into the rules and the score went to 100%. What gives the shortcut away?skill

07In three sentences

GoodhartA metric that becomes the target gets optimized by any path, including memorizing and tampering with the meter.
Meter outCases and measurement script frozen with a hash, in a manifest the agent can't reach.
Hidden casesKeep only when the gain also shows up on cases the agent has never seen.

Going further

Darwin Gödel Machine (arXiv)LOOP-R: critique and feasibility, Goodhart (docs/06)Course 1: frozen tests, shortcuts, and a calibrated judgekit/hashes.py
Previous lessonNext lesson