📝 Prompt as specification
In automation, a prompt isn’t a conversation — it’s natural language code. It will run on its own a thousand times, with no one there to correct the AI in real time. That's why it needs to be a specification: precise, unambiguous instructions that define exactly the expected behavior.
✓ Specification prompt
- ✓Defines the role, task, and exact output format.
- ✓Cover edge cases ("if there’s no data, return X").
- ✓Behaves the same way every time it runs.
✗ Conversational prompt
- ✗"Help me analyze this" — vague and dependent on context.
- ✗Doesn’t say what to do when something goes off script.
- ✗Gives different answers each time.
💡 Practical tip
Before writing the prompt, ask: if I gave this instruction to a new employee, could they carry it out without asking me anything? If the answer is no, the prompt still has gaps.
Runs on its own a thousand times.
Needs to be right first.
Cover the unexpected.
Same input, same output.
🏗️ Structure of a reliable prompt
A fixed skeleton makes behavior predictable: role (who the AI is), context (what it needs to know), task (what to do), output format (how to respond) and rules (what to respect). Below, the same pattern applied to an order classifier.
🧾 Structured prompt example
📐 Why named sections work
- •AI separates instructions (rules) from data (the customer's message) without confusing them.
- •You review and version each section independently.
- •Becomes a template: the same framework works for any classifier.
Who the AI is.
What you need to know.
How to respond.
What to respect.
🧾 Structured output (JSON) and validation
Free-form text breaks automation: the next step doesn't know where each piece of information is. Enforcing JSON with fixed fields e validate the response before using it is what makes the AI output reliable enough to trigger an automated action.
📤 Expected JSON output
✅ Validation before use (pseudocode)
⚠️ Attention
Never blindly trust the AI’s output. If the JSON comes back malformed or with an invented category, validation should reject and retry (or escalate)—never let an invalid value proceed to an automated action.
Fixed, named fields.
Before using the output.
Values within the allowed range.
Invalid → retry or escalate.
🚧 Guardrails and fallback
AI makes mistakes and hallucinates. Guardrails those are the limits — what it can't do, what validation rejects. Fallback is what happens when it isn’t sure: ask for confirmation, escalate to a human, or use a safe default. Together, they prevent an error from becoming an irreversible action.
Prompt guardrail
Explicit instructions about what never to do: don’t invent categories, make decisions outside the scope, or mark confidence as low when uncertain.
Validation guardrail
A programmatic check that rejects outputs outside the format or list of values—the AI proposes, the code verifies.
Confidence-based fallback
When confidence falls below a threshold (e.g., 0.6), the workflow routes to human review instead of acting on its own.
💡 Practical tip
Ask the AI for a field confidence for every output and use it as a fallback trigger. It’s the simplest way to turn "the AI isn’t sure" into a concrete flow decision: act on its own only above the threshold; escalate below it.
What AI can’t do.
Field that triggers a fallback.
Human when uncertain.
An error doesn’t trigger an irreversible action.
🔀 Branching logic based on AI
This is where AI stops being decoration and becomes the brain: a categorical field in the JSON output (e.g., category or priority) becomes the key to a IF/switch in the workflow, routing each case to a different action. AI classifies, the workflow executes.
📐 Rules for a healthy branch
- •Use a field categorical (closed list) as the key—never free text.
- •Always have a default branch for the case that doesn't match any option.
- •Combine with confidence: the right category plus low confidence can still be escalated.
Workflow runs.
Closed categorical field.
Branch for the rest.
Category + confidence.
🔁 Test and iterate on the prompt
A prompt that gets it right in one example can fail in ten. Build a set of test cases — including the difficult and adversarial ones—run the prompt against all of them, measure the accuracy rate, and adjust until it stabilizes. Only this cycle gives you the confidence to let AI make decisions on its own in production.
Gather real-world cases
Use real messages from the process (from the T2 audit), including ambiguous ones and those that tend to cause confusion.
Define the answer key
For each case, note what the correct output would be. This answer key is the yardstick you use to measure the prompt.
Run, measure, adjust
Run the prompt on all cases, compare the results with the answer key, calculate the accuracy rate, and adjust the prompt where it failed. Repeat until it stabilizes.
✓ Healthy iteration
- ✓Includes difficult and adversarial cases from the start.
- ✓Measures the accuracy rate before and after each adjustment.
- ✓Stop iterating when the rate stabilizes, not on the first attempt.
✗ Naive iteration
- ✗Tests with just one easy example and calls it done.
- ✗Adjusts based on "guesswork," without measuring whether it improved.
- ✗Fixes one case and breaks three without noticing.
Multiple cases, not just one.
Adversarial cases first.
Accuracy rate, not guesswork.
Iterate until it stops improving.
🎯 Module Summary
Next Track:
Track 4 · Build — move from the blueprint to building the end-to-end automation.