π Spec Document as Input
At the advanced level, the input is no longer a plan.md with 5 tasks. It's a complete spec document that describes an entire application. The orchestrator reads the spec, divides it into rounds, and dispatches workers to build everything.
"We send in our initial spec to the orchestrator and it has to reason about that and then figure out how many workers to spin off."
π Spec Example (Kanban Board)
# Spec: Single Page Kanban Board
## Objetivo
Criar um Kanban board como web app estatico (HTML + CSS + JS).
## Requisitos
- 3 colunas: To Do, In Progress, Done
- Drag and drop entre colunas
- Persistencia via localStorage
- Responsivo (mobile-first)
- Tema dark com cores amber/neutral
## Criterios de Aceitacao
- App carrega sem erros no browser
- Drag and drop funcional
- Dados persistem apos reload
- Layout correto em mobile (320px+)
Cole Medin demonstrated exactly this scenario: "building a single page Kanban board as a static web app, I just take this prompt and I'll show you it running live right now." The orchestrator received the spec and automatically split the work into rounds.
π― The Orchestrator Plans Rounds
The orchestrator analyzes the spec and decides on the strategy: how many rounds, how many workers per round, and which tasks each worker gets. This is the system's most critical decision, which is why it requires a strong model.
"The orchestrator is deciding how to split up the work right now. The orchestrator spent 6,000 tokens with that initial planning and then prompting our first three workers in round number one."
π Multi-Agent Orchestrator Prompt
You are the orchestrator for a multi-agent build system.
Input: spec.md (attached)
State store: Postgres (Neon) at DATABASE_URL
Your job:
1. Read spec.md and break it into independent work units
2. Plan rounds: group units that can run in parallel
3. For each round, dispatch workers with specific prompts
4. After each round, read worker results from state store
5. Validate: does the output match spec requirements?
6. If validation passes, plan next round. If fails, retry or escalate.
7. When all spec requirements are met, report completion.
Pause for human approval before starting each new round.
β Workers Execute and Report
Each worker receives a specific task from the orchestrator and operates in own session with an isolated worktree and database branch. On completion, it writes the results to the state store (Postgres), not the context window.
"Those workers are going to go back and they're going to update the state that we have in our database. And so this is our loop, right? Because then the next time the orchestrator runs, it's going to get that updated state from the workers."
π Prompt Template for a Worker
You are Worker {{worker_id}} in Round {{round_number}}.
Your task: {{task_description}}
Spec context: {{relevant_spec_section}}
Previous round results: {{previous_results_summary}}
Instructions:
1. Execute the task in your worktree
2. Run validation: {{validation_command}}
3. Report results to state store:
- Status: success/failure
- Files changed: list
- Test results: pass/fail count
- Notes: any issues encountered
Do NOT modify files outside your assigned scope.
β Orchestrator Validates and Launches the Next Round
After each round, the orchestrator reads the results from the state store, checks whether the spec criteria have been met, and decides whether to launch another round or declare completion. This is the checkpoint that distinguishes a robust system from a blind loop.
"Itβll move forward one round at a time, validating each one. We can also have a person review what happened in the first round before the orchestrator moves on to the next."
π§ Validation Logic Between Rounds
After Round {{N}} completes:
1. Query state store for all worker results in this round
2. Check: did all workers report success?
- If any failed: retry failed workers (max 2 retries)
- If still failing: pause and escalate to human
3. Check: do combined results satisfy spec requirements?
- Run integration tests across merged worktrees
- Compare output against spec acceptance criteria
4. If all good: plan Round {{N+1}} with remaining work
5. If spec is fully satisfied: merge all branches, report done
π Human-in-the-Loop Checkpoints
Human-in-the-loop (HITL) is what separates a reliable system from an experiment. Without HITL, you risk letting the loop run for hours and coming back to find that the result is unusable.
"A lot of times, people set these systems up to just run, run, run, run. Then you let them run for a day. When you come back, all you find is garbage."
π Where to Place Checkpoints
After planning
Review the rounds planned by the orchestrator before it starts dispatching workers.
Between rounds
Validate the previous round's results before authorizing the next one.
Before the final merge
Review all changes before merging into the main branch.
π― Reflection Point
Cole Medin sums up the philosophy: "That is the kind of reliability that I feel like we really need to have right now in order to build anything more than simple demos." HITL isnβt optional in production systems. Itβs what turns a demo into something usable.
β Questions
The multi-agent orchestrator is the peak of complexity. Questions that come up in real-world use.
Is the cost justified?
Cole reports using more than 1M tokens for a single simple application. Multi-agent multiplies that. The honest answer: it depends on the value of the output versus the cost of a human developer. It may be worthwhile for PoCs. For production, calculate the cost first.
When complexity becomes the enemy
Not every project needs a multi-agent orchestrator. Cole is clear: "I would just fold loop engineering into harness engineering." If a simple plan.md does the job, don't scale up to multi-agent just because it seems more sophisticated.
Orchestrator model vs. worker model
Cole uses Pi/Kimi for the orchestrator (routing decisions) and Claude for workers (actual implementation). The orchestrator doesn't need the most expensive model, but it does need reliable reasoning. Test before deciding.
π― Reflection Point
Multi-agent is a powerful tool, but it isn't a universal solution. Use it when the problem truly requires parallelism, rounds, and validation between stages. For most scenarios, the issue bot (4.2) or even the task runner (4.1) is enough.
π Module Summary
Next Module:
4.4 -- Anti-Patterns and Troubleshooting (what goes wrong and how to fix it)