PTENES
Skip to content
MODULE 4.3

🔭 Failures, harness, and observability

Here’s the cost of the graph: fan-in that swallows a source, a cycle that doesn’t converge, state that leaks from one node to another. This module shows the three failure modes that exist only because you built a graph—and how to instrument it so you don’t discover each one in production.

6
Topics
45
Minutes
Advanced
Level
Practice
Type
Progress in this module
0%0 of 6
1

🔀 Find the merge that swallows a source

A fan-in node—the point where several parallel branches come back together at a single node—has a silent failure mode: one branch fails or returns empty, and the join moves on as if it were complete. The result looks polished, but it’s incomplete, and no one notices because nothing raised an alarm.

🆕 New here? Two words before you continue

  • Fan-in / merge / join: the node where several branches that ran in parallel (fan-out) come back together. It’s the opposite of fan-out.
  • Cardinality: expected number of items. If you dispatched 4 branches, the expected cardinality at the join is 4. Wrong cardinality = something disappeared along the way.

The most common way this happens: you dispatch research across four sources, one API goes down, that source’s node returns an empty list (not an error—empty), and the merge writes the report with three sources, thinking there are four. Nobody got an exception, nobody saw a red log. The report is just wrong.

source A source B source C ✗ source D junction (merge) 3 of 4 sources arrived — merge didn't check, and continued as if complete

What to look at: the red dashed line from source C—it still reaches the join, just empty. The merge node doesn’t check the quantity, so it processes what it received and continues. No error is raised. The defect isn’t in the source that failed; it’s in the join that didn’t ask, "Did I receive all four that I dispatched?"

✓ Safe join

  • ✓Counts how many items arrived and compares that with how many were dispatched
  • ✓Explicitly decide what to do with a missing branch: abort, continue while marking the gap, or retry
  • ✓Records in the output "3 of 4 sources, C unavailable" when you decide to proceed anyway

✗ Blind join

  • ✗Merges “what arrived” without knowing how much was expected
  • ✗Treats an empty list the same as a list with data—both “pass” in the merge
  • ✗No log or metric records the gap

Key concepts

Fan-in

Where parallel branches come together

Cardinality

Did I receive the N I dispatched?

Silent failure

Empty isn’t an error, but it should be

Explicit decision

Abort, flag, or try again

2

♾️ Break the Routing That Doesn’t Converge

Two conditional edges pointing to each other are the most common recipe for a stuck graph: the reviewer rejects, the writer rewrites, the reviewer rejects again — indefinitely. The graph “has been running” for hours, and the symptom is exactly that: no error, just consumption.

💡 What every cycle needs to have

  • •Iteration cap — a maximum number of repetitions, period, even if nobody has "solved" anything
  • •Progress criterion — two iterations without measurable improvement = give up and alert; don’t keep trying the same way
  • •Without both, the bill is infinite—you pay in tokens, time, and credits until someone notices manually
NO LIMIT — it hangs writer reviewer ∞ rejected → rewrite → rejected → rewrite... WITH A CEILING — converges writer reviewer round 2 of 3 give up and warn you

What to look at: the entire difference is in the third edge on the right, the cyan one, which exists only because someone designed an emergency exit. Without it, the graph on the left has only two edges—and both can only point to each other.

⚠️ Warning: “eventually converges” is not a plan

If your defense against an infinite loop is “the reviewer will approve it eventually,” you have no defense at all. Write down the explicit number (e.g., max 3 rounds) AND the progress criterion (e.g., the quality score has to go up each round) — both, not just one. A limit without a progress criterion lets the graph use all 3 rounds even when the second already showed it won't improve.

Key concepts

Iteration cap

Maximum number, no exceptions

Progress criterion

2 rounds with no improvement = stop

Emergency exit

Third edge out of the pair

Infinite bill

No cap = no cost limit

3

🚰 Seal the state leak

State leakage is when a node passes along more than it should, and the next node’s clean context is no longer clean. Clean context was exactly the advantage that motivated building the graph—and this is the failure mode that silently cancels out that advantage without visibly breaking anything.

The classic example is the reviewer. If the reviewer receives only the text and the approval criteria, they judge it with fresh eyes. If the reviewer receives the reasoning the writer’s thought process too—“I thought about it this way, then decided that”—it stops judging the result and starts validating the process. In practice, it goes back to approving the work, because it now understands (and empathizes with) the reason behind the choice.

lean state writer reviewer text + criteria reviewer judges with fresh eyes leaked state writer reviewer text + criteria + reasoning stamps the work again

What to look at: only the edge's content changed — not the graph's structure. That's what makes state leakage hard to spot in code review: the diagram looks identical; only the payload traveling along the edge has grown. Audit edges by schema, not just by looking at the diagram; it’s the only reliable way to catch this.

📊 Where leakage tends to slip in unintentionally

  • Forgotten debug log in the payload — you left the writer’s “thinking out loud” in the output field
  • Context shared for convenience — a single session object that every node reads and writes
  • Entire conversation history — instead of just the summary, the next node receives the raw transcript
  • “Just in case” fields — someone adds an extra field "might be useful later" without checking whether it should leak

Key concepts

State leakage

An edge carries more than it should

Clean context

The advantage the graph promised

Stamp on return

A contaminated reviewer always approves

Silent failure

Same diagram, different payload

4

📐 Write the state schema

State schema is the explicit declaration of what each edge carries: field names, each field’s format, what’s required, and—just as importantly—what’s explicitly no must pass. It’s the contract between two nodes. Without a contract, changing one side breaks the other without warning, and nobody knows who’s responsible.

1

Required fields

The minimum the next node needs to work—without it, it shouldn’t even start.

E.g.: texto e criterio_aprovacao to a reviewer.

2

Forbidden fields

The part most people forget to write down. Without it, “just to be safe” becomes the default.

E.g.: raciocinio_do_escritor Do NOT pass it to the reviewer—that's topic 3 state leakage.

3

Format and validation

The field name alone isn't enough—you need to specify the type and, if possible, validate it before passing it on.

E.g.: score is a number between 0 and 1, not the string “eighty percent.”

💡 Practical tip

Write the state schema BEFORE writing the prompt for each node—on paper or in a sample JSON. If you can’t clearly name the fields, that’s a sign the two nodes are still conceptually mixed together, and perhaps should be one.

Key concepts

State schema

The contract for each edge

Required vs. prohibited

Both sides need to exist

Before the prompt

The schema defines the node, not the other way around

No shared blame

Clear contract = traceable failure

5

💾 Make execution durable

Durable execution is a graph’s ability to survive a failure along the way: resume exactly where it left off, not redo a node that already finished successfully, and be idempotent — running the same input twice produces the same result, with no duplicated side effects.

This isn't something new invented by the agent world. It's exactly what data orchestration has solved for decades with durable execution and per-node caching (think of data pipeline tools that resume a stuck job without rerunning steps that have already completed). Agentic graphs inherit this ready-made solution — there's no need to reinvent it.

✓ Durable execution

  • ✓Each completed node saves the result + checkpoint before moving on
  • ✓Resuming after a failure starts at the next pending node, not from the beginning
  • ✓Same input + same graph = same path (reproducible)

✗ Fragile execution

  • ✗The entire graph runs again from scratch after any failure
  • ✗That has already sent an email or written a file repeats the action on retry (not idempotent)
  • ✗No record of “how far I’d gotten”

📊 Three questions to check whether your graph is durable

  • If it crashes at node 4 of 7, what happens on restart? — if the answer is "run everything again," it isn't durable
  • Does running a node twice change the real-world result? — if so (duplicate email, double charge), it isn't idempotent
  • Can you reproduce exactly the same path with the same input? — otherwise, you can't really debug it

Key concepts

Durable execution

Survives a failure midway through

Idempotent

Repeating doesn't change the result

Cache per node

Doesn't redo what's already finished

Inherited from data

Orchestration already solved this

6

🖥️ Instrument it or debug in the dark

Observability here it’s simple: the graph harness needs to show which nodes ran, in what order, with what latency and cost. In a loop, “why did it do that?” gets buried in a transcript (the raw conversation/execution log) that you read line by line. In a graph, the path taken is data you can inspect — if instrumentation exists.

Without this instrumentation, you pay the full cost of the graph—more nodes, more edges, more failure modes— and don’t get the advantage that justified it: seeing the system. And instrumentation isn’t optional in a graph the way it was “desirable” in a loop; without it, you literally have no way to answer “where did it get stuck?”

execution monitor — graph:daily_brief#run_4821
✓ coletar_fonte_a1.2s · $0.003
✓ coletar_fonte_b0.9s · $0.002
✗ coletar_fonte_c4.0s · timeout
⧗ reviewer (turn 3 of 3)running · $0.041 total
○ juncao_finalwaiting

illustrative recreation, not a screenshot of a real tool.

What to look at: the red line (coletar_fonte_c) is the warning from topic 1 appearing on your screen—without this panel, that timeout becomes the silent "3 of 4 sources" that no one noticed. The amber line shows the cycle from topic 2 on the third and final allowed turn, with the accumulated cost visible—you see the limit approaching before it blows the budget.

🧪 Practical example: graph audit prompt

Objective: give Claude Code (or another assistant with file access) a grafo.json describing nodes and edges, and asking it to find the five flaws in this module before you put it into production.

Voce vai auditar o arquivo <grafo.json> deste projeto, que descreve um grafo de
agentes (nos, arestas, condicoes). Procure especificamente estes cinco defeitos
e reporte cada ocorrencia com o no/aresta exato:

1. CICLO SEM TETO DE VOLTAS
   Existe algum par de arestas condicionais que formam um ciclo (A -> B -> A)
   sem um campo tipo "max_voltas" ou equivalente? Se sim, aponte o ciclo.

2. ARESTA CONDICIONAL SEM "QUANDO"
   Toda aresta que sai de um no de decisao/roteamento declara a condicao sob a
   qual e seguida? Aponte arestas condicionais sem condicao explicita.

3. CONDICOES QUE NAO COBREM TODOS OS CASOS (ENTRADA ORFA)
   Some as condicoes de saida de cada no de decisao. Existe um caso possivel
   (ex.: nem aprovado nem reprovado) que nao tem aresta correspondente? Isso
   deixa o fluxo sem destino nesse caso.

4. JUNCAO SEM CHECAGEM DE CARDINALIDADE
   Todo no que recebe de mais de uma aresta (fan-in) tem alguma logica de
   checagem de quantidade/completude antes de seguir? Aponte juncoes que nao
   verificam se receberam todos os ramos esperados.

5. NO SEM TETO DE CUSTO
   Todo no que chama um modelo tem um limite de custo, tokens ou tempo
   declarado? Aponte nos sem nenhum limite.

Para cada defeito encontrado: cite o id do no/aresta, explique o risco em
1 frase, e sugira a correcao minima.

How to verify: run this prompt against a graph you know is broken — pick one grafo.json valid and deliberately remove the field max_voltas of a cycle. If the assistant doesn’t identify this exact cycle as defect 1, the prompt is too loose — rewrite it and ask it to list ALL cycles before deciding whether there’s a cap.

Now swap it for yours: point to your own <grafo.json> real. If more than one defect appears, prioritize fixing the unbounded cycle and the unchecked join first — those two can turn into an infinite bill or incomplete data without warning the fastest.

Quick check (doesn’t block anything): the reviewer starts approving everything after you began including the writer’s reasoning in the edge leading to it. What’s the flaw?

Key concepts

Observability

Which nodes, in what order, how much it cost

Transcript

Raw log that you read line by line

Path as data

You look it up; you don't guess

No instrument

Pay the cost, lose the advantage

📌 Module Summary

✓
A join needs to check cardinality — did it receive the N items it dispatched? Explicitly decide what to do with anything missing.
✓
Every cycle needs a cap and a progress criterion — without both, the bill is infinite.
✓
State leakage cancels out the graph — contaminated context makes the reviewer rubber-stamp it again.
✓
State schema is the contract — both required and prohibited; both sides need to exist.
✓
Durable and idempotent execution — resumes without redoing work, repeats without duplicating effects.
✓
Without observability, you pay and get nothing back — the path taken must be data you can query.

What this covers:

The graph isn’t free. You get parallelism, clean context, and an auditable flow—and pay for it with a state schema, new failure modes, and execution/observability infrastructure. If you’re not going to instrument it, you probably shouldn’t have built the graph.

Next Track:

Track 5 — Hands-on: migrate a real loop to a graph, model business and task graphs, and take home a library of ready-to-use prompts.