E19 · The two alerts
10 minRun the rehearsal.
Done when: ALERTA: ciclo 3 usou 88% do teto, ALERTA: 3 erros seguidos iguais, and saída: 1.
Lesson 7 · 18 min
Observability: traces and cost per cycle
A loop that runs on its own must leave a trace: how long each cycle took, how many tokens it spent, what it decided and why. Without it, an error loop or a cycle that blows your quota only shows up at month's end. In this lesson you record one line per cycle and use it to get cost per cycle, cost per keep, and the alerts.
Start the lesson
In this lesson you will
- 1One JSON line per cycle: the minimum worth keeping
- 2Cost per cycle and cost per keep
- 3Alerts: ceiling, cycles without a keep, repeated identical errors
- 4Real tokens from a Claude Code call
Goal: record a trace per cycle (time, tokens, verdict) and use it to get cost per cycle, cost per keep, and the meta-agent's alerts.
01From the cycle to the alert
The same role as the LOOP-R Meta-agent: it watches the system, not the problem, and only reports. Click each block.
A trace without tokens is still useful: time, status, and reason already show an error loop and stagnation.
The alert does not stop the loop: what stops it is the course 2 ceiling (loop-longrun.sh) or the person. Here, it is reading the trace.
02Rehearsal: six cycles, two problems
Fake agents with made-up tokens: one expensive cycle and three identical failures (like an agent with no response because of the session limit).
Terminal · cost-free rehearsal
K=~/projetos/loop-r-eng-aprende-v72/kit
E=${TMPDIR:-/tmp}/ensaio-aprende/a7; rm -rf $E; mkdir -p $E
bash $K/novo-sistema.sh $E/sis > /dev/null; export RASTRO=$E/rastros.jsonl
c() { TOKENS_FALSOS="$2" AGENTE="bash $K/agente-falso.sh $1" bash $K/experimento.sh $E/sis "$1" > /dev/null; }
c comenta "8000 500"; c sinonimos "9000 700"; c estraga "41000 3000" # made-up tokens for the rehearsal
c falha "0 0"; c falha "0 0"; c falha "0 0"
echo "## uma linha do rastro"; sed -n 2p $RASTRO
echo "## custo por ciclo"; python3 $K/custo-ciclo.py $RASTRO --teto 50000; echo "saída: $?"Example from the repository
## uma linha do rastro
{"ciclo": 2, "descricao": "sinonimos", "segundos": 0, "tokens_entrada": 9000, "tokens_saida": 700, "status": "keep", "motivo": "melhorou"}
## custo por ciclo
ciclo status segundos tokens(ent+saída) motivo
1 discard 0 8500 não melhorou
2 keep 0 9700 melhorou
3 discard 0 44000 não melhorou
4 erro 0 0 agente falhou (saída 1): ERRO: sem resposta do m
5 erro 0 0 agente falhou (saída 1): ERRO: sem resposta do m
6 erro 0 0 agente falhou (saída 1): ERRO: sem resposta do m
total: 6 ciclos · 1 keep · 62200 tokens
custo por keep: 62200 tokens
ALERTA: ciclo 3 usou 88% do teto de 50000 tokens
ALERTA: 3 erros seguidos iguais: agente falhou (saída 1): ERRO: sem resposta do modelo (limit
saída: 1The tokens in this rehearsal are made up (the TOKENS_FALSOS variable); time, status, and alerts come from the real script. The error loop is the case that costs the most without showing up: in course 1, the real judge stopped at the session limit from the 16th call on.
03The tokens of a real call
agente-claude.sh asks claude -p --output-format json and writes to the trace the sum of input (with cache) and output. One call through your subscription.
Terminal · real, one call
K=~/projetos/loop-r-eng-aprende-v72/kit
E=${TMPDIR:-/tmp}/ensaio-aprende/a7 # the rehearsal above
RASTRO=$E/rastros.jsonl AGENTE="bash $K/agente-claude.sh" bash $K/experimento.sh $E/sis "claude"; echo "saída: $?"
tail -1 $E/rastros.jsonl | python3 -c 'import json,sys; d=json.load(sys.stdin); print("tokens:", "sim" if (d["tokens_entrada"] or 0) > 0 else "não", "·", d["status"])'Example from the repository
ciclo 7 · keep · 0.800 → 1.000 · melhorou saída: 0 tokens: sim · keep
The line written to the trace in this run: 10 seconds, 247,492 input tokens (adding cache reads and writes) and 364 output tokens. The request is under 2 KB: almost all of the input is the context that the Claude Code session loads on this machine. One measurement, Claude Code 2.1.303; yours changes with your instructions, plugins, and MCPs.
Estimating the loop's cost from the size of the request. The calculation that counts is the trace's, per cycle, with the entire context the tool loads. Cache and resume cost are covered in course 2, lesson 4.
04What is already measured at INEMA
Two local cases, with the date of each number.
1. LOOP-R example
Each cycle of vendas-whatsapp has one custo.md per agent. The four cycles add up to about 1.7 million tokens for one promotion (the example's README).
Synthetic sales data, real agents: the cost is real, the effect was designed.
2. Local telemetry
Claude Code hooks write events to a local SQLite (session, tool, error, subagent) and a dashboard shows what is running. Measured: 3 to 5 thousand tool calls per day, about 10 thousand events and 2 MB per day.
The project's golden rule: store names and numbers, never the text of a prompt, response, or command.
3. OpenTelemetry
There are OpenTelemetry conventions for agents (agent spans, model-call spans, and tool spans, with tokens), described as experimental in secondary sources.
Before adopting, check the status in the OpenTelemetry documentation. One JSON line per cycle already covers what this lesson asks for.
05Practice
Rehearsal, real, and your loop.
E20 · Real tokens
5 minRun the real block right after the rehearsal.
Done when: the last printed line starts with tokens: sim; if it comes out as tokens: não · erro, the agent got no response (session limit): wait and run again.
E21 · Trace in your loop
20 minHook RASTRO into your experiment (or write the same JSON line at the end of each cycle of your loop) and set the per-cycle token ceiling from the real measurement in E20.
Done when: python3 $K/custo-ciclo.py [your trace] --teto [your ceiling] lists one cycle per line and ends in sem alerta or in an ALERTA: that you can explain.
06Lesson quiz
Question 1 of 4
Six cycles, one keep, 62,200 tokens in total. What is the cost per keep?skill
Correct
Total divided by the keeps: 62,200 / 1. The cost per keep includes the attempts that didn't stay.Not that one
Total divided by the keeps: 62,200 / 1. The cost per keep includes the attempts that didn't stay.Review: Rehearsal: six cycles, two problemsQuestion 2 of 4
Three cycles in a row end in an error with the same reason. What is this usually?skill
Correct
A bad hypothesis gives a discard, not an error. The same repeated error is infrastructure, and every round costs time and quota.Not that one
A bad hypothesis gives a discard, not an error. The same repeated error is infrastructure, and every round costs time and quota.Review: From the cycle to the alertQuestion 3 of 4
What should the trace keep?
Correct
Numbers are enough for cost and alerts, and prompt and response text may hold sensitive data. This is the INEMA local telemetry rule.Not that one
Numbers are enough for cost and alerts, and prompt and response text may hold sensitive data. This is the INEMA local telemetry rule.Review: What is already measured at INEMAQuestion 4 of 4
In the real call that was measured, the request was under 2 KB and the input went over 247 thousand tokens. Why?
Correct
Instructions, memory, and the rest of the harness go into every call. That is why you measure cost in the trace, not in the size of the request.Not that one
Instructions, memory, and the rest of the harness go into every call. That is why you measure cost in the trace, not in the size of the request.Review: Rehearsal: six cycles, two problemsQuiz complete
You got {a} of {n} right. Get them all right and the lesson is marked as complete.Choose an answer. The explanation appears after you try.
07In three sentences
One lineEach cycle records time, tokens, status, and reason; no prompt text.
Cost per keepTotal tokens over the improvements that stayed; it includes the attempts that didn't stay.
AlertsCeiling, cycles without a keep, and repeated errors in a row; the alert reports, and the ceiling and the person stop.