🎭 The Orchestrator: The System’s Brain
At the heart of any scalable loop engineering system is the orchestrator: the agent session that reads the specification, decides which tasks to execute, launches workers, and validates results. It doesn't write code directly. Its role is to reason about the project's state and make high-level decisions about what to do next.
💬 How Cole Medin Describes the Orchestrator
"We send in our initial spec to the orchestrator and it has to reason about that and then figure out how many workers to spin off. And then it has to prompt them all. They each spend tokens and then the results come back."
The orchestrator acts like an automated project manager. It receives the spec (what needs to be done), analyzes the repository's current state, and decides: How many workers do I need? Which tasks should I assign to each one? In what order? When is the work complete? Each of these decisions costs reasoning tokens, which is why the orchestrator is one of the most expensive parts of the system.
📋 Orchestrator Responsibilities
- ✓Read the spec and understand the full scope of the work
- ✓Plan rounds: divide the work into execution waves
- ✓Dispatch workers: create specific prompts for each worker
- ✓Validate results: check whether the workers delivered what was expected
- ✓Decide the next step: launch a new wave or declare completion
⚙ Workers: Parallel and Isolated Execution
Workers are independent agent sessions that receive a specific task from the orchestrator and execute it. Each worker operates in isolation, ideally in its own git worktree, so multiple workers can work in parallel without stepping on each other's code.
"If multiple agents are handling tasks in a loop, we need to make sure they run in isolation so they don’t get in each other’s way. That’s how we scale our output with AI-assisted coding."
✓ Worktrees for Isolation
- ✓Each worker operates in an isolated copy of the repository
- ✓No file conflicts between parallel workers
- ✓Each one can commit and create independent branches
- ✓Merge controlled by the orchestrator at the end
⚙ Anatomy of a Worker
- 1.Receives a specific task from the orchestrator
- 2.Starts an agent session in a worktree
- 3.Executes: writes code, runs tests, validates
- 4.Report the result to the state store
💾 External State Management
One of the most critical problems with long-running loops is the context bloat: as the agent accumulates conversation history, the LLM's context window becomes overloaded and response quality degrades. The architectural solution is to move state outside the LLM, using an external database as the source of truth.
"When you're just using slash loop in Claude Code, it's really just continuing in the same coding agent session. So if you loop for a while, you're going to completely bloat your context for your LLM and overwhelm it."
📊 Context Window vs. State Store
✗ No External State
The entire history lives in the context window. After 10–20 iterations, the LLM starts to "forget" initial instructions, generate duplicate code, and lose coherence. It's like trying to memorize an entire book instead of consulting notes.
✓ With External State (Postgres/Neon)
The database stores completed tasks, pending tasks, validation results, decision logs, and cumulative cost. Each new orchestrator session reads only the current state, not the entire history. The context window becomes cache, not storage.
🗄 What the State Store Stores
Runs
Execution history: original spec, rounds completed, overall status (running, paused, completed, failed).
Events / Logs
Every orchestrator decision and every worker result. Enables auditing and post-execution debugging.
Task State
Status of each individual task: pending, in_progress, completed, failed. The orchestrator reads this to decide the next round.
Cost Tracking
Tokens consumed per round and per worker, plus cumulative total cost. Lets you set budget limits.
🔄 The Full Flow of a Round
A "round" is a unit of work of the orchestrator-workers system. Each round follows a predictable flow: the orchestrator reads the state, decides what to do, dispatches workers, collects results, and evaluates whether more rounds are needed. This cycle repeats until the goal is reached or the budget runs out.
🏃 Step-by-Step Guide to a Round
Orchestrator reads the state store
Query the database to find out: which tasks are pending? What did the workers from the previous round deliver? Are there errors to fix?
Plans the worker wave
Decides how many workers to launch, which tasks to assign to each one, and builds the specific prompts. This is fan-out.
Workers run in parallel
Each worker starts in an isolated worktree, executes its task (writes code, runs tests), and writes the result to the state store.
Fan-in: orchestrator collects results
When all workers finish, the orchestrator reads the results from the state store and evaluates: did everything pass? Are there failures? Do I need to rerun any workers?
Decision: next round or completion
If work remains, the orchestrator plans the next round. If the goal has been reached, it marks the run as complete and reports to the human.
"The orchestrator spent 6,000 tokens with that initial planning and then prompting our first three workers in round number one."
🛡 Durability and Resumability
Long loops inevitably fail at some point. The machine may restart, the agent session may crash, the network may fluctuate. The question isn't if it will fail, but when. Durability means the system can resume where it left off without losing work already done. And this is only possible with external state management.
"Estou armazenando aqui todos os loops que executamos, os diferentes eventos e logs, para que possamos sempre retomar um workflow mais tarde. Assim, gerenciamos todo o nosso estado em um banco de dados externo."
🔧 How Resumability Works
Checkpoint per round
At the end of each round, the full state is saved to the database: completed tasks, worker results, and orchestrator decisions. If the system goes down between rounds, just read the latest checkpoint and continue.
Resume from last good state
The orchestrator doesn't need conversation history to resume. It reads the state store and knows exactly where the project stands. A new agent session is just as effective as the previous one.
Cleanup of orphaned workers
If a worker crashes mid-execution, the orchestrator detects it (via timeout or database status) and can redispatch the task to a new worker in the next round.
🎯 Reflection Point
Without durability, loop engineering is an expensive toy. Imagine running a loop for 4 hours, spending hundreds of thousands of tokens, and losing everything because the terminal closed. External state turns a fragile experiment into a system that can actually run. Cole Medin built exactly that in his dashboard: "I'm storing this here so we can always resume a workflow later on."
❓ Questions
The orchestrator-workers architecture is elegant on paper, but raises serious practical questions. Cole Medin, despite building a functional dashboard for it, takes a cautious stance.
Is the orchestrator a cost bottleneck?
Each orchestrator decision consumes reasoning tokens. The more complex the spec, the more tokens it takes to plan rounds and evaluate results. Cole showed that a single run of a simple app consumed more than 1 million tokens. The question is: is it worth paying that reasoning cost for every decision, or does a deterministic workflow (like Archon) work better?
Who validates the validator?
The orchestrator validates the workers' results, but the orchestrator is also an LLM. If it hallucinates during validation (accepting a bad result as good), the error propagates to subsequent rounds. Without a human in the loop at critical points, self-validation can become a dangerous blind spot.
Determinism vs. autonomy: where's the balance?
Cole Medin prefers deterministic workflows where a human defines the steps. Boris Cherny prefers full autonomy, with the orchestrator deciding everything. The truth is probably somewhere in the middle: use the orchestrator for routing and model decisions for code, but not to define the entire process.
⚠ Cole's Challenge
Cole sums up his position on giving the orchestrator full autonomy: "We want to actually take the decision away from the coding agent as much as we can, only applying the reasoning of the LLM when we actually need it to write the code." The orchestrator should do the minimum necessary, not the maximum possible.
📋 Module Summary
Next Module:
2.1 — Real Advantages (Track 2: Advantages vs. Disadvantages)