🪜 3-tier hierarchy
Invalidation follows a 3-tier hierarchy: one change
invalidates the its tier and the ones below it. Since the order is
tools → system → messages, making more changes at the beginning has a wider blast radius.
What it is
The three tiers are cascading layers. The model is the root: changing it invalidates everything. Next come tools, system, and messages, each invalidating itself and everything after it.
Why learn
Because it lets you calculate the "blast radius" of any change before making it: changing a message is cheap; changing tools or switching models costs the entire prefix.
The earlier in the conversation (tools) the change happens, the bigger the cascade. A new message at the end costs almost nothing.
Key concepts
| Change | What invalidates it |
|---|---|
| switch model | everything (tools + system + messages) |
change tools | everything (tools + system + messages) |
change system | system + messages |
| change message | messages only |
🎚️ tool_choice / thinking on-off
Not everything that changes per request breaks the cache. Changing tool_choice each request, or turn it on/off thinking,
keeps the tools+system cache. These are parameters that
don’t enter the cacheable prefix — you don’t need to worry about them.
What it is
tool_choice e thinking are request behavior controls, not prefix bytes. Changing them doesn’t change the prefix match.
Why learn
Because it prevents paranoia. It's easy to think that "anything I change per request breaks the cache"—and then you stop using useful features out of unfounded fear.
✓ Can vary freely
- ✓
tool_choicedifferent for each request. - ✓Turn on/off
thinking.
✗ Keeps knocking it out
- ✗Change the list
tools. - ✗Edit the system prompt.
Key concepts
tool_choice doesn’t enter the prefix.
thinking on/off preserves tools+system.
What brings it down is content of the prefix.
No need to panic: not every parameter breaks.
🔭 20-block lookback
Each breakpoint looks for the previous entry, checking at most 20 blocks
back. A turn that adds more than 20 blocks — loops with
many tool_use/tool_result —
can make the next turn silently miss the cache.
What it is
The lookback is the breakpoint’s search window: it only finds the cached prefix if it’s within the previous 20 blocks. Beyond that, it won’t find it — even if the cache exists.
Why learn
Because it's the "invisible" cause of misses in agents that make many tool calls per turn. Nothing changed in the prefix, but the breakpoint doesn't reach the previous entry.
Illustrative recreation. What matters: in long loops, insert an intermediate breakpoint before exceeding 20 blocks.
Practical tip
If your agent makes many tool calls per turn and the cache "disappears" even though the prefix hasn't changed, suspect the lookback. A intermediate breakpoint every ~15 blocks keeps the entry within the window.
Key concepts
Lookback = 20 blocks backward.
Turn with >20 blocks = risk of a miss.
Common in loops of tool_use/tool_result.
Fix: a breakpoint every ~15 blocks.
👻 Silent invalidators
Silent invalidators are bytes that change in the prefix without you noticing.
The usual suspects: datetime.now()/Date.now()/uuid
in system; json.dumps without sort_keys=True;
iterate set; user-assembled tools; conditional section in the system
(if flag: system += ...).
What it is
These are sources of nondeterminism that change the prefix on every request, even when the cache is warm. Unlike a cold write (caused by time), the problem here is that the content changes.
Why learn
Because they are the #1 cause of cache that "never works." The symptom is always the same: cache_read_input_tokens always 0.
✗ Common silent invalidators
- ✗
datetime.now()/Date.now()/ uuid in system. - ✗
json.dumpswithoutsort_keys=True. - ✗Iterate
set(variable order). - ✗Tools assembled by the user.
- ✗Conditional section in system (
if flag: system += ...).
✓ Corrections
- ✓Inject date/time into the
messages, not in the system. - ✓Serialize with
sort_keys=True. - ✓Sort with
sorted(...)before assembling. - ✓Freeze the system; move dynamic content to the end.
Symptom
cache_read_input_tokens always 0 repeating the same prefix. It’s the telltale sign that a silent invalidator is active.
🌐 Concurrent timing (fan-out)
The cache entry only becomes readable AFTER that the 1st response starts streaming. So N requests parallel with the same prefix pay all at full price — none has seen the other’s cache yet.
What it is
During fan-out (sending several agents/requests at once), the first request hasn't finished writing the prefix when the others start. They all write from scratch.
Why learn
Because firing everything at once seems faster, but throws away the shared cache. A small change in launch order saves a lot.
Fix: send 1, wait for the 1st token (the cache becomes readable), and only then fire off the N-1 — which now read cheaply.
Key concepts
Readable input only after the 1st stream.
N parallel requests with the same prefix = all 1×.
Fix: 1 → wait for the 1st token → N-1.
Applies to fan-out / parallel subagents.
🧪 Verify with usage
The measurement instrument is the object usage in the response:
cache_read_input_tokens e
cache_creation_input_tokens.
Zero on repeated requests = has an invalidator. Then you
byte diff of the prompt between two identical requests to find the byte
that changes.
What it is
Audit with usage is to compare the cache fields between two requests that should be identical. If the read doesn’t increase, something in the prefix is changing.
Why learn
Because it's the difference between guessing and proving. With the byte diff, you can pinpoint the exact silent invalidator—the date, the uuid, the out-of-order key.
Copy-run example — audit silent pipeline invalidators
Objective: have Claude Code scan the code that builds the prompt and list every silent invalidator + how to fix it.
Audite o pipeline em <caminho da pasta/arquivo que monta
o prompt>. Procure especificamente, no que alimenta o
PREFIXO (tools + system), por:
- datetime.now() / Date.now() / uuid interpolados no system
- json.dumps(...) SEM sort_keys=True
- iteração de set (ordem não-determinística)
- montagem de `tools` a partir de set/dict sem sorted(...)
- seções condicionais no system (if flag: system += ...)
Para cada ocorrência, aponte:
1. arquivo e linha;
2. por que quebra o cache (muda os bytes do prefixo);
3. a correção (mover o dinâmico para as messages,
sort_keys=True, sorted(...), congelar o system).How to verify: after applying the fixes, run the same request twice in a row and compare the usage. O cache_read_input_tokens must stop being 0 on the 2nd request. If it’s still 0, do the byte diff of the prompt between the two requests to find what changed.
Key concepts
usage = measurement instrument.
Repeated Read 0 = active invalidator.
A byte diff pinpoints the exact change.
Prove it, don’t guess: always measure.
Module summary
usage.Next track:
Track 3 — Advanced best practices (caching vs. compaction, cache-friendly architecture, ready-to-use prompts)