🔧 Technical — the cache in action
Now that the terms are clear, let's look at the cache actually working: how it behaves in a growing conversation, what appends without breaking and what resets position 0, and invalidation in depth — with examples copy-run to run in your Claude Code.
The cache isn’t a battery that runs down: the portion read from the cache grows every turn. A turn with something new (e.g., a large skill) counts less; the rest becomes a cheap read on the next turn.
Everything in this track comes from a single invariant: prompt caching is a
prefix matching. The cache key is the exact bytes of the prompt
up to the cutoff, and the assembly order is always
tools → system → messages. Remember this phrase — it
explains why appending at the end is safe and why changing the tool list breaks everything.
The three fields you'll audit all the time: cache_read_input_tokens
(read from cache, ~0.1×), cache_creation_input_tokens (written, ~1.25×)
and input_tokens (only the uncached remainder, at full price). The
sum of the three is the prompt size. If the read field stays
zero repeating the same prefix, there’s a silent invalidator.
Trail map
Detailed content
💬 The cache in a conversation
The stable prefix grows, each turn rereads it cheaply, and the cache % GOES UP — it doesn’t run down. Where the cold write comes in and why a long pause costs you.
system + tools + the entire history (including the last response) form the stable prefix, which grows with each turn.
It’s what makes the conversation cheap: the larger the stable prefix, the more tokens cost ~0.1×.
Prefix = an identical beginning that repeats; only the new ending misses the cache.
With each new message, only the end changes; the API rereads the entire prefix from the cache (~0.1×) and writes only the new block.
Explains why the second turn onward is much cheaper than the first.
Hit = cache read; write = recording the new block for that turn.
As long as you respond within ~5 min, the window renews and the cache stays warm.
Working at a steady pace keeps everything cheap with little effort.
TTL = how long the entry lives; each use restarts the 5 min clock.
A turn can show 40% (there was something new, e.g., a large skill). That block becomes a cheap read on the next turn, and the percentage goes up.
Corrects the mistaken intuition that the cache “runs out”; it only gets better as the conversation continues.
Current turn: 40k read / 60k new = 40%; next turn: ~100k read / ~1k new ≈ 99%.
The 1st message writes to the cache (costs a little more); there is no previous prefix to read.
Prevents the shock of seeing an "expensive" first turn — it's the only one that writes everything.
~1.25× write (5 min TTL); starting with the 2nd turn, ~0.1× reads predominate.
After sitting idle for ~5 min, the TTL expires, and the next message pays to write it again.
Explains why resuming after lunch "costs": it was a cold write, not a bug.
For traffic with gaps > 5 min, there's a 1 h TTL (write ~2×).
🧩 Skills and tools: append vs. break
New content at the FIM is safe; changing the list of tools resets position 0. Why Tool Search deliberately appends.
(a) The skill DESCRIPTION (the short line in the list) always stays in the stable prefix. (b) The full CONTENT is only included when the skill is invoked.
Without separating the two, you think “every skill is expensive”—only the content adds weight, and only once.
Description = always present and cheap; content = on demand.
The skill content is APPENDED to the END of the history (not at position 0). Everything before it remains valid and is read cheaply; only the new block is written.
Takes the fear out of invoking skills mid-conversation: it’s a single write, not a reset.
Appending at the end = safe; on subsequent turns, the skill is already a cheap read.
The tools are rendered at position 0 (before system and history). Changing the list tools invalidates everything from that point on.
It’s the most expensive silent invalidator: one byte at position 0 invalidates the entire prefix.
Assembly order: tools → system → messages.
(1) Connect/disconnect an MCP midway; (2) change description/input_schema from a tool; (3) assemble tools of a set/dict without sorting; (4) enabling/disabling tools by mode/permission.
These are the 4 most common causes of a cache that “disappears” for no apparent reason.
Changing just one word in a description is enough to invalidate position 0.
Tool Search loads a tool’s schema by APPENDING it (not replacing the list) — specifically to AVOID breaking the cache.
Distinguishes “finding a tool via Tool Search” (safe) from “connecting an MCP that rewrites the list” (breaks it).
Appending the schema at the end = one write; rewriting tools = reset.
New content at the FIM = safe. Changing the list of tools or in the system prompt = position 0 reset.
One sentence that resolves 90% of everyday decisions about the cache.
Always ask: "does this change the beginning, or just add to the end?"
🔎 Invalidation in depth
3-tier hierarchy, the 20-block lookback, silent invalidators, and concurrent timing in the fan-out. How to audit with usage.
A change only invalidates its tier and those below it: changing tools OR switching models = full reset; changing system = invalidates system+messages; changing a message = messages only.
You calculate the “blast radius” of any change before making it.
Tiers in order tools → system → messages; the model is the root of everything.
Replace tool_choice per request or turn it on/off thinking keeps the tools+system cache.
Avoids paranoia: not every parameter that changes per request breaks the cache.
These two aren't part of the cacheable prefix — no need to worry.
Each breakpoint looks for the previous entry, checking at most 20 blocks back. A turn that adds >20 blocks (loops with many tool_use/tool_result blocks) may silently cause the next turn to miss the cache.
It’s the “invisible” cause of cache misses in agents that make many tool calls per turn.
Fix: an intermediate breakpoint every ~15 blocks.
datetime.now()/Date.now()/uuid in system; json.dumps without sort_keys=True; iterate set; user-assembled tools; conditional section in the system.
These are changes you don’t “see” that alter the prefix bytes on every request.
Common symptom: cache_read_input_tokens always 0.
The entry only becomes readable AFTER the 1st response starts streaming. N parallel requests with the same prefix all pay the full price.
Sending everything at once may seem fast, but it throws away the shared cache.
Fix: send 1, wait for the 1st token, fire off the N-1.
cache_read_input_tokens / cache_creation_input_tokens in the response. Zero on repeated requests means there’s an invalidator.
It’s the measuring instrument—without it, you’re guessing; with it, you can prove it.
Diff the prompt bytes between two identical requests to find the byte that changes.