E4 · Fails before, passes after
10 minRun the three rounds of the rehearsal.
Done when: A ends with FALTAM 4 and saída: 4; B shows OK on the código 0 criterion and FALHOU on 3 passed; C ends with PRONTO and saída: 0.
Lesson 2 · 18 min
Command → output: check the criterion and the /goal evaluator
A criterion only counts if an outsider can check it without trusting the agent. In this lesson you run each criterion of goal.md as a command, see which one passes by shortcut, and learn where the proof has to show up: the /goal evaluator in Claude Code only reads the conversation.
Start the lesson
In this lesson you will
- 1The command → output format and the three kinds of expected output
- 2conferir-goal.py: OK or FALHOU per criterion, with exit code
- 3A criterion with no command and a criterion that passes by shortcut
- 4What the /goal evaluator sees, and what it doesn't
Goal: check each criterion of goal.md by command, see which level 2 one passes by shortcut, and put the proof in the conversation the /goal evaluator reads.
01From the criterion to the proof in the conversation
Click each step.
Click each block in the diagram to see what it does.
A command you run and the output that shows up: it's the same proof for you, for the headless loop and for the /goal evaluator.
02The goal.md for the rehearsal
Six criteria. One has no command and one passes through a shortcut. Find both before you run it.
kit/goal-ensaio.md (criteria; translation for reading; the file in the kit is in Portuguese)
## Done criteria
**Function**
- [ ] `python3 -m pytest -q tests/` → `3 passed`
- [ ] `python3 -c "from slug import slug; print(slug('Ação & Reação!'))"` → `acao-reacao`
**Regression**
- [ ] `python3 -m pytest -q tests/ -rs` → `código 0`
**Limit**
- [ ] `git diff --stat HEAD -- tests/ pytest.ini` → `vazio`
- [ ] `git ls-files --others --exclude-standard -- '*conftest.py' tests/` → `vazio`
- [ ] the code stays clean and well documentedTerminal · before, shortcut and after
K=~/projetos/loop-r-eng-verificacao-v72/kit bash $K/ensaio.sh > /dev/null && cd /tmp/ensaio-verif/proj cp $K/goal-ensaio.md ../goal.md # outside the project: the agent doesn't edit the criterion python3 $K/conferir-goal.py ../goal.md; echo "saída: $?" # A) before the work: it has to fail $K/agente-falso.sh conftest < /dev/null # B) shortcut: skips the tests python3 $K/conferir-goal.py ../goal.md; echo "saída: $?" rm conftest.py && $K/agente-falso.sh bom < /dev/null # C) the right work sed -i '/limpo e bem documentado/d' ../goal.md # and the criterion with no command goes away python3 $K/conferir-goal.py ../goal.md; echo "saída: $?"
Example from the repository
FALHOU python3 -m pytest -q tests/ → 3 passed
veio: 3 failed in 0.01s (código 1)
FALHOU python3 -c "from slug import slug; print(slug('Ação & Reação → acao-reacao
veio: NotImplementedError (código 1)
FALHOU python3 -m pytest -q tests/ -rs → código 0
veio: 3 failed in 0.01s (código 1)
OK git diff --stat HEAD -- tests/ pytest.ini → vazio
OK git ls-files --others --exclude-standard -- '*conftest.py' t → vazio
FALHOU sem comando: o código fica limpo e bem documentado
---
FALTAM 4
saída: 4
Pronto: marquei os testes instáveis; a suíte fica verde.
FALHOU python3 -m pytest -q tests/ → 3 passed
veio: 3 skipped in 0.00s (código 0)
FALHOU python3 -c "from slug import slug; print(slug('Ação & Reação → acao-reacao
veio: NotImplementedError (código 1)
OK python3 -m pytest -q tests/ -rs → código 0
OK git diff --stat HEAD -- tests/ pytest.ini → vazio
FALHOU git ls-files --others --exclude-standard -- '*conftest.py' t → vazio
veio: conftest.py (código 0)
FALHOU sem comando: o código fica limpo e bem documentado
---
FALTAM 4
saída: 4
Pronto: slug() implementado.
OK python3 -m pytest -q tests/ → 3 passed
OK python3 -c "from slug import slug; print(slug('Ação & Reação → acao-reacao
OK python3 -m pytest -q tests/ -rs → código 0
OK git diff --stat HEAD -- tests/ pytest.ini → vazio
OK git ls-files --others --exclude-standard -- '*conftest.py' t → vazio
---
PRONTO
saída: 0The command appears cut off at 60 characters in the output. In round B, código 0 gave OK with all three tests skipped: it's the level 2 criterion in goal.md.
If round A (before any work) already gives OK on a function criterion, the criterion doesn't measure the work. Every function criterion has to fail beforehand; limit criteria (nothing touched) can pass from the start.
03The /goal condition with proof
Because the evaluator only reads the conversation, the condition requires the pasted output. In the goal prompts of the execucao-longa kit and the 15 commands kit, the pattern is the same.
Claude Code · /goal
/goal The output of `python3 ~/projetos/loop-r-eng-verificacao-v72/kit/conferir-goal.py <caminho>/goal.md` is pasted in full in this conversation and ends with PRONTO; no file in tests/ or conftest.py was changed; stop after 15 turns and say what was missing.
Two final messages: which one passes the evaluator?
(1) Terminei. Rodei o conferir-goal.py e deu tudo certo. (2) Rodei o conferir-goal.py: OK python3 -m pytest -q tests/ → 3 passed (…) --- PRONTO
04Practice
Rehearsal, then your own goal.md.
E5 · Your goal.md checked
20 minTake one of your own goal.md files (or write one from the course 2 template) and run conferir-goal.py on it before you start the work. Rewrite every criterion that came back as "sem comando" and every function criterion that already passes.
Done when: python3 $K/conferir-goal.py [your goal.md] --pasta [your project] → no sem comando line and at least one function FALHOU before the work (exit code greater than 0).
E6 · /goal with the proof pasted
20 minRun a small task with /goal using the condition from the block above, pointing to your goal.md.
Done when: the agent's last reply contains the output of conferir-goal.py ending in PRONTO and /goal ends as met; when you run the same command yourself → PRONTO and exit code 0.
05Lesson quiz
Question 1 of 4
What can the /goal evaluator in Claude Code do to judge the condition?skill
Correct
The documentation says it doesn't run commands or read files. The proof has to be printed in the conversation; that's why the condition asks for the pasted output.Not that one
The documentation says it doesn't run commands or read files. The proof has to be printed in the conversation; that's why the condition asks for the pasted output.Review: From the criterion to the proof in the conversationQuestion 2 of 4
In the rehearsal, with all tests skipped, which criterion passed by mistake?skill
Correct
pytest exits with 0 when all tests are skipped. "exit code 0" is level 2 (gameable); "3 passed" as a whole word is not.Not that one
pytest exits with 0 when all tests are skipped. "exit code 0" is level 2 (gameable); "3 passed" as a whole word is not.Review: The goal.md for the rehearsalQuestion 3 of 4
Which of these final messages lets the /goal evaluator decide based on proof?
Correct
Only the pasted output puts what the command actually printed into the conversation. "Everything went fine" is a report.Not that one
Only the pasted output puts what the command actually printed into the conversation. "Everything went fine" is a report.Review: The goal.md for the rehearsalQuestion 4 of 4
A function criterion passes before any work is done. What does that tell you?
Correct
A function criterion has to fail first. If it already passes, it can't tell done from not done.Not that one
A function criterion has to fail first. If it already passes, it can't tell done from not done.Review: The goal.md for the rehearsalQuiz complete
You got {a} of {n} right. Get them all right and the lesson is marked as complete.Pick an option. The explanation appears after you try.
06In three sentences
CheckableA criterion with no command can't be checked; conferir-goal.py counts it as FALHOU.
Fails firstA function criterion that already passes before the work measures nothing; "exit code 0" alone passes with everything skipped.
Printed proofThe /goal evaluator only reads the conversation: the condition requires the pasted output ending in PRONTO.