🤖 Why a single model has a ceiling
Before bringing several models together, it helps to understand what one. An LLM (Large Language Model) is the type of AI that runs behind ChatGPT and Claude: it receives text and returns text, predicting the next word based on everything it read during training. Each model has a profile — one is better at code, another at writing, another at long-form reasoning.
The consequence: No model is best at everything. There’s a “ceiling” for each one. The orchestration bet is that if you combine the best of each one, you might surpass the ceiling any model could reach alone. Maybe. The entire course is about testing that “maybe.”
🆕 New here? Three terms
- LLM: The AI model that generates text (the “brain” of ChatGPT/Claude).
- Token: the small piece of text the model reads and writes (≈ ¾ of a word). You pay per token — both the ones that go in and the ones that come out.
- Tools: actions the model performs on its own — search the web, run code, edit a file.
❓ The 2 questions that define everything
There’s a simple way to understand any multi-LLM system, no matter how complex it seems. It answers two questions, and nothing else. If you can answer them for a system, you understand the system.
Memorize this equation. Fugu, Fusion, and their sub-agents only change how answer each side — never the questions.
🎯 The two questions
- 1.Who handles each part? — o routing. It could be your rule, a workflow, or a model trained to choose.
- 2.How are the answers combined? — a merging. Concatenate, vote, have a judge choose, or have a model rewrite everything into a single text.
🎼 Maestro, specialists, and merger
Every orchestration has three roles. The conductor (or “conductor”) reads the request and divides up the work. The specialists are the models that execute each part. And the merger combines the answers into a single result. Recognizing these three roles makes any “magical” system easy to understand.
Conductor (left) → specialists in parallel (cyan) → merger (right). This is the skeleton of Fugu—and, turned sideways, of Fusion.
Maestro / conductor
Reads the request, decides whether to handle it alone or split it up, and chooses who gets each part. This is where “coordination intelligence” lives.
Specialists
The models that do the work: one good at coding, another at reasoning, another at writing. Each handles the part it’s good at.
Fuser
Combines the answers into a single result—by concatenating them, choosing the best one, or rewriting everything. Without a merger, you have N separate answers, not one.
📦 "Mixture of experts" as 1 API
“Mixture of experts” is the idea of keeping several specialists and calling on the right one for each case. The commercial trick is package this behind a single API: you hit a single endpoint, send your request, and the whole team runs behind the scenes. From the outside it looks like one model; inside, it's an orchestra.
📊 What "1 API" hides
The abstraction is elegant, and it has a price. You gain simplicity; you pay in control and money:
- You don't see which models were called or how many.
- You pay the tokens from all of them, not just the final answer.
- You wait the whole team — hiring, running, and merging takes time.
✓ What 1 API gives you
- ✓Zero orchestration code
- ✓One endpoint, one billing
- ✓Hedge: don’t get locked into a single vendor
✗ What it requires
- ✗Hidden team token costs
- ✗Combined latency across multiple models
- ✗Little visibility into what happened
🛠️ You already orchestrate (without realizing it)
Orchestration isn’t exotic. When Claude Code opens sub-agents (workers that can run different models—haiku, sonnet, opus) and divides a plan among them, it’s orchestrating: one conductor divides the work, several execute in parallel, and one brings it all together. The only difference from Fugu is who decides and crosses providers.
💡 Practice: orchestrate it yourself
Goal: watch the orchestration (fan-out + fusion) happen right in front of you, in Claude Code.
Use uma abordagem multi-agente neste arquivo: abra 3 sub-agents em PARALELO, cada um revisando por um ângulo diferente (1: bugs, 2: performance, 3: legibilidade). Depois um agente final FUNDE os três achados num só relatório priorizado. Arquivo: <caminho/do/seu/arquivo>
How to verify: you’ll see 3 agents running at the same time (the fan-out) and, at the end, a single consolidated report (the fusion). Switch <caminho/do/seu/arquivo> for a real file of yours.
🧾 The hidden cost of orchestration
Here’s the counterweight that determines whether the math works out. Each specialist the maestro calls reads and writes tokens — and you pay for all of them. Add the wait time: hiring, running, and merging the team takes much longer than a model responding directly. More models = a bigger bill and more time.
📊 A taste of the numbers (Track 2)
In the field test you’ll study in depth in Track 2, the orchestrator (Fugu) versus a strong model on its own (Opus 4.8), on the same tasks:
- 4.5× slower overall
- 5× more expensive (US$ 53,60 vs US$ 10,66)
- 36 of 38 tasks: technical tie
⚠️ The mental trap
Without seeing the hidden cost, “adding more models” always seems better. It isn’t. A team is only worth it when it delivers something you can’t achieve alone — and, as you’ll see, that’s rarer than the hype suggests.
Quick recall (optional): orchestration answers two questions. What are they?
✅ Module summary
Next module:
1.2 — The spectrum: from human to conductor model