🔀 Provider Mix: Pi/Kimi + Claude
The first and most impactful cost lever Cole demonstrates is don’t use the same model for everything. In the Agent Control Plane dashboard, he runs the entire orchestrator with Kimi (via Pi), reserving Claude just for actual code implementation.
"I’m running everything with Pi. In fact, I’m using my Kimi subscription with Kimi K 2.6, now Kimi K 2.7, to run all these workflows. So, yeah, that’s a lot of tokens, but I’m not using Opus for everything. I’m still getting great results because of the harness I built here, which makes the model more capable."
🧩 The Principle: Harness Raises the Model’s Capabilities
Cole’s insight is that a well-built harness (deterministic workflow, external state, validation at each step) lets you use cheaper models without sacrificing final quality. The harness compensates for the model’s limitations by providing precise context and well-defined steps.
Orchestrator
Kimi K 2.7 / Pi
Planning, round decisions
Implementation
Claude Code
Writing real code
Review
Codex / lightweight model
Validation and code review
📊 Model Routing: The Right Model for Each Task
Model routing goes beyond provider mix: it’s about route each specific task to the most suitable model in terms of cost-effectiveness. Cole demonstrates this in Archon, where each workflow node can use a different model.
"Posso decidir qual modelo usar em cada nó deste fluxo de trabalho do Archon. Por exemplo, na etapa de classificação, podemos usar um modelo pequeno, como Haiku, Minimax M3 ou Kimi K 2.7."
🎯 Routing by Complexity
Classification (low cost)
Decide whether an issue is a bug or a feature. Haiku, Minimax M3, or Kimi K 2.7. Simple reasoning, cheap tokens.
Exploration and research (medium)
Investigate the codebase, understand dependencies. Kimi K 2.7 with long context. Requires reasoning, but not maximum precision.
Implementation (expensive)
Write real code that will become a PR. Claude Code. Quality matters here, and you pay for it.
💡 The Single-Model Anti-Pattern
Cole points out directly: "one of the other big issues I see with using slash goal or routines or loops in Cloud Code is you're just using one model for pretty much everything. That's part of the problem of why it's so expensive." Using Opus to classify a bug versus a feature is like taking a truck to the corner store.
💾 External State: Avoiding Context Bloat
Context bloat is the silent killer of loops. When you run /loop in the same Claude Code session, each round accumulates context in the LLM's context window. After many rounds, the model gets overloaded and quality drops sharply. The solution is to keep state outside the LLM.
"If you leave it looping for a while, your LLM’s context gets huge, and it gets overloaded. That’s why we need a system that distributes the work across different coding agent sessions."
No External State
- ✗Context window grows with each round
- ✗Quality degrades progressively
- ✗If the session crashes, everything is lost
- ✗Token costs grow exponentially
With External State (Postgres)
- ✓Each session starts fresh
- ✓Orchestrator reads only what it needs from the database
- ✓Crash? Resume from the last recorded state
- ✓Token cost stable per round
The Paradigm: Context Window as Cache
The mental shift is to treat the LLM’s context window as temporary cache, not permanent storage. The database (Postgres/Neon) is the storage. Each agent session loads what it needs from the database, executes, and writes back. The context window never accumulates the history of previous rounds.
🌳 Worktrees for Isolating Parallel Sessions
When you scale loops to process multiple tasks in parallel, agents can't share the same working directory. Git worktrees create isolated copies of the repository where each agent works without interfering with the others.
"Work trees are also a really important part of loop engineering. Boris talks about this as well. If we're having many different agents handling tasks in a loop, we need to make sure they're running in isolation so they're not stepping on each other's toes."
Three-Layer Isolation
1. Code: Git Worktrees
Each worker runs in a separate worktree with its own branch. Changes in one worker don't affect the others. Merging happens only after validation.
2. Data: Neon Database Branching
Neon lets you create database branches, just as Git does with code. Each worker can have its own data branch for testing without affecting production.
3. Gates: Conflict Avoidance
When multiple agents run dev servers, each needs a different port. Without port management, parallel agents conflict and crash.
⏱ Batch vs. Real-Time: When Each Makes Sense
Not every loop needs to run in real time. Accumulate tasks and process them in batches can be significantly cheaper than processing each task immediately. The choice depends on urgency and volume.
Batch
- ✓When: Accumulated GitHub issues, overnight maintenance, pending code reviews
- ✓Advantage: the orchestrator plans once for N tasks, amortizing the planning cost
- ✓Trade-off: higher latency; results aren’t immediate
Real-Time (immediate)
- ✓When: critical bug in production, feature blocker, urgent hotfix
- ✓Advantage: immediate response, fast result
- ✓Trade-off: higher cost per task; the orchestrator plans for each one
🎯 Reflection Point
Cole demonstrates the batch pattern by dispatching 4 GitHub issue workflows simultaneously. The orchestrator plans once, dispatches them all, and validates afterward. This is more efficient than processing each issue individually because the orchestrator’s planning cost is amortized across the 4 tasks.
❓ Questions and Essential Metrics
Optimizing costs without metrics is guesswork. Cole built cost tracking into the dashboard precisely to have concrete data on where the money is going. Three metrics are the minimum required for any loop setup.
📐 The Three Essential Metrics
1. Tokens per task
How many tokens each individual task consumes, including orchestrator overhead. If a simple task consumes 100k tokens, something is wrong. Comparing similar tasks reveals anomalies.
2. Cost per round
The cost of each orchestrator-workers cycle. If it rises round after round, that indicates context bloat or an inefficient orchestrator. If it stays stable, the system is healthy.
3. Success rate
Percentage of workers that complete successfully versus fail. If the rate falls below 80%, the effective cost per useful task skyrockets because you're paying for discarded work.
Is it worth optimizing?
It depends on volume. If you run 1 loop per week, cost optimization isn't a priority. If you run dozens a day like Boris, every cent per token matters. The decision to invest in optimization should be proportional to usage volume.
Will the cost decrease naturally?
Models get cheaper over time. Kimi K 2.7 is already dramatically cheaper than Opus for many tasks. But “cheaper” doesn’t mean “cheap.” 1M tokens, even with an inexpensive model, still represent a real cost. The techniques in this module matter regardless of the price per token.
The final cost rule
Cole sums up the approach: "you got to have the right system. Otherwise, things are going to completely fall apart." Cost isn't just about money; it's also about reliability. A cheap loop that produces junk costs more than an expensive loop that delivers value.
📋 Module Summary
Next Module:
4.1 - Simple Loop: Task Runner - ready-to-use recipe with a copyable prompt and plan.md template