Theme

Source

Size

Text width

Between the lines

Control accent

0 of 0 0%
MODULE 3.2

Observation and supervision

Track the agent with reports, a failure log, and triggers, and keep your judgment with you.

At the end: Build the report, the failure log, the triggers, and the supervision ritual for an agent, and turn every mistake into a rule.

6 topics
50 min estimate with hands-on practice
6 commented exercises
1 final check
1

The report you want to read

Day numbers Cases for you Errors Fixed schedule
The report always has the same parts in the same order, and the errors section exists even when it’s empty.

What it is

A report is the summary the agent delivers at a scheduled time, saying what it did. A good report has a few simple numbers (how many sent, confirmed, pending), the list of cases it sent to you for decisions, and the list of errors. It fits on one screen and takes two minutes to read. The golden rule: error is never hidden—it always appears in its own section.

Why learn

Without a report, you only find out what the agent did when something goes wrong and someone complains. With a report, you see the problem the same day. A report that only shows success is worse than none: it creates false confidence. It’s from this that you decide whether the agent moves up or down a level.

Key concepts

Simple numbers3 to 5 counts you can understand at a glance.
Cases passed to youwhat needs human decision-making.
Errorsalways in its own section, even if it’s “none.”
Fixed scheduleevery day or every week, always at the same time.
COMMENTED EXAMPLE · 3.2.1
Goal: ask the agent for a short, honest report.

REPORT — deliver <every day at 17:00>
1. Numbers: <sent · confirmed · did not respond · errors>
2. Passed to you: <name, 1-line reason>
3. Errors: list them all, even the small ones. If none, write “No errors today”.
4. Questions: what you couldn’t figure out.
Never omit an error so the report looks better.

How to check: intentionally trigger an error (a signup with an empty phone number) and see if it shows up in section 3 of the report.

✓ Do this

✓ Require the errors section to always be present, with “no errors” written when that’s the case.

✗ Avoid this mistake

✗ Ask for a long, pretty report no one reads, or one that only shows what went right.

Practice before you reveal

Write the 4 counts you’d like to see in the daily report of a coordinator of volunteers at an NGO.

View commented answer

Something like: confirmed shifts, rejections, substitutes found, and shifts still unfilled. The last one is the most important, because it’s the one that requires your action before the weekend ends. Good numbers point to a decision, not just volume.

2

Failure log: one line per error

Date What broke Smallest correction Rule changed
Each error goes through the same four columns, and the last one links the error to a concrete change in the agent sheet.

What it is

The failure log is a simple table where each agent error becomes a row: date, what broke, the smallest correction, and which line in the Agent Sheet was changed. A log—technical name for this kind of record—is just that: a dated note of what happened. It’s not an incident report or meeting minutes; it’s one line, written on the spot, with no narrative.

Why learn

After about ten lines, the pattern appears on its own: the errors cluster around the same rule, the same source, or the same type of client. Without a log, each error looks new and you end up rewriting everything. With a log, you see that a single protection would have been enough—like a cap, an extra check, or an additional prohibition. It’s also the evidence to move the agent up or down a level.

Key concepts

Datewhen the error was noticed.
What brokethe fact, in one sentence, without blame.
Smallest correctionthe smallest change that prevents the repetition.
Rule changedwhich line in the agent sheet changed.
COMMENTED EXAMPLE · 3.2.2
Goal: keep the agent’s failure log in a one-line-per-error table.

| date | what broke | smallest correction | sheet rule changed |
|---|---|---|---|
| <12/09> | <sent reminder to Saturday patient> | <check day of the week before sending> | <Rules: “Saturday has no appointments” became mandatory verification> |
| <15/09> | <replied with an old service price> | <use only the current table> | <Source: “printed table doesn’t count”> |

How to check: every line has all 4 columns filled in; a line without “rule changed” means the error could still happen again.

From concept to action

  1. Date: identify the starting condition.
  2. What broke: apply the decision described.
  3. Smallest correction: check the effect in the example.
  4. Rule changed: record the output evidence.

✓ Do this

✓ Write the line on the same day as the error, before moving on to the next task.

✗ Avoid this mistake

✗ Store the errors in memory and try to remember them only during the monthly review.

Practice before you reveal

The store agent promised a customer a delivery date that wasn’t in the system. Write the log line.

View commented answer

Example: 18/09 · promised delivery date outside the system · only inform dates that are on the orders panel · Prohibitions earned “promising a date that is not in the system”. Notice that the correction is small and verifiable; it’s not “improving service”.

3

Triggers: when it pauses and calls you

This situation happens Is it a trigger? Pauses and calls you You decide
The trigger interrupts the normal flow and returns the decision to the human before the error gets out of control.

What it is

Triggers are the situations where the agent stops what it’s doing and calls you immediately, without waiting for the report. This is escalation, also called escalade: handing the case to whoever has authority to decide. Good triggers are concrete and observable, such as “client threatening to complain,” “drop above 50%,” or “two patients at the same time.” Each Agent Sheet should have 3 to 5 triggers.

Why learn

The report shows what happened; the trigger prevents the worst from happening before the report. Without triggers, the agent tries to solve on its own situations that require judgment—and that’s where serious complaints are born. Working triggers are also the criterion for reaching N4: without tested alarms, the whole process can’t be in the agent’s hands.

Key concepts

Concretesomething the agent can recognize without interpreting.
Urgentcan’t wait for the end-of-day report.
Clear actionit pauses, informs you, and doesn’t try to solve it.
Testedeach trigger went through a test before turning on.
EXAMPLE WITH COMMENTS · 3.2.3
Goal: paste the trigger list into the agent instruction and test the first one.

STOP AND CALL ME IMMEDIATELY when:
- <Irritated customer or threatening to complain>
- <Refund request>
- <Product with a defect>
How to call: <message in my WhatsApp with the customer name and a 2-line summary>.
While waiting: tell the customer “<I’ve passed this to the responsible person; they will reply to you today.>” and promise nothing.

How to check: simulate “I’m going to complain to Procon” and confirm you receive the alert and the agent stops negotiating.

✓ Do this

✓ Write triggers the agent recognizes by facts, and test at least the first one before turning it on.

✗ Avoid this mistake

✗ Use vague triggers like “delicate situation,” that the agent can’t identify.

Practice before you reveal

Write three triggers for an assistant that makes the first round of corrections to a teacher’s essays.

View commented answer

Good examples: suspected plagiarism or text made by AI, concerning content like bullying or a risk to the student, and writing that’s off-topic. All three require human judgment and have consequences for the student; the agent flags and stops, it doesn’t decide.

4

The ritual: 2 minutes a day, 10 per week, 1 review per month

Before turning on Every day · 2 min Every week · 10 min Every month · level
The ritual goes from most frequent and short to rare and strategic, and each frequency has a different question.

What it is

Supervision isn’t just watching the agent work; it’s a short, fixed ritual. Every day: 2 minutes—read the report, handle the cases sent to you, and log errors in the failure log. Every week: 10 minutes—re-read the errors and adjust one single rule in the Agent Sheet. Every month: ask whether the level is still right and whether the task is still the same. Before turning it on, an extra step: the 3 tests passed, and the team knows there is an agent.

Why learn

Without a ritual, supervision happens only when something explodes—and by then it’s too late. With a ritual, the cost of supervision stays small and predictable, fitting into a business owner’s schedule. Adjusting one rule per week, not five, lets you know which change solved the problem.

Key concepts

Before turning on3 tests, updated source, team informed.
Every day2 minutes in the report and past cases.
Every week10 minutes and an adjusted rule.
Every monthis the level and the task still the right ones?
EXAMPLE WITH COMMENTS · 3.2.4
Goal: print and use the agent supervision checklist.

BEFORE YOU CALL
☐ The 3 tests passed
☐ The official source is up to date
☐ Everyone knows there’s an agent doing this

EVERY DAY (2 minutes)
☐ I read the report (<every day at 17:00>)
☐ I resolved the cases it called me about
☐ Any errors? I noted one line: date · what happened · the smallest correction

EVERY WEEK (<Friday, 10 minutes>)
☐ I reviewed the errors noted from the week
☐ I adjusted ONE sheet rule (no more than one)
☐ Did the success criterion get met?

EVERY MONTH
☐ Is level <N2> still correct?
☐ Is the task still the same? If it changed, redo the sheet.

How to check: at the end of the month, count how many daily squares stayed empty. More than five means the ritual doesn’t fit your routine and needs adjustment.

✓ Do this

✓ Mark on your calendar the report time and the day of the weekly review, as a fixed commitment.

✗ Avoid this mistake

✗ Change multiple rules at the same time after a bad week.

Practice before you reveal

Pick an agent of yours and set the exact time for the daily report and the day of the weekly review.

View commented answer

The best time is right after the agent finishes its shift, when you can still act on the same day. The weekly review works best on a calm day, like Friday morning or early Monday. If you can’t name the time, supervision doesn’t exist yet.

5

Supervision is a human responsibility

Objectives and limits Agent executes You audit and decide exceptions You are responsible for the outcome
The agent sits in the middle of the process, but the ends—defining and responding—stay human.

What it is

The agent executes, analyzes, and even decides within the limits of the Agent Sheet—but the responsibility for the outcome remains yours. Supervision here is the management function: set objectives, establish limits, evaluate results, correct failures, decide exceptions, and evolve the process. Auditing—checking by sampling that the work is correct—is part of that. The agent expands your capacity; it doesn’t replace your judgment.

Why learn

When something goes wrong with a client, “it was the AI” isn’t an acceptable answer for anyone: not for the client, not for the law, and not for you. Knowing what can’t be delegated prevents handing the agent decisions that require human context—like an exception for an old client or a sensitive case with a student. Your role shifts from doing everything to deciding what matters.

Key concepts

Judgmenton exceptions, new cases, and dilemmas stays with you.
Auditingchecking by sample, even when the report is clean.
Limitsonly you raise the level or remove a prohibition.
Responsibilitythe outcome answers in your name.
EXAMPLE WITH COMMENTS · 3.2.5
Goal: write down what stays with you and do a weekly audit by sample.

WHAT IS NOT DELEGATED IN THIS AGENT
- Deciding exceptions: <old customer asking for special timing>
- Changing limits: <raise level, remove a prohibition>
- Sensitive cases: <formal complaint, legal question>

WEEKLY AUDIT (5 minutes)
1. Pick <5> items at random from the week’s work.
2. Check each one against the official source.
3. Note: <5> checked · <0> with errors · error found goes to the failure log.

How to check: if the audit finds an error that wasn’t in the report, the report is hiding failures and needs to be fixed first.

From concept to action

  1. Objectives and limits: identifying the initial condition.
  2. Agent executes: applying the described decision.
  3. You audit and decide on exceptions: check the effect in the example.
  4. You are responsible for the outcome: record the output evidence.

✓ Do this

✓ Check a sample of the work every week, even when the report says everything is fine.

✗ Avoid this mistake

✗ Let the agent decide exceptions because it “already knows the client.”

Practice before you reveal

List two decisions from your business that you would never hand to an agent, even at N4, and explain why.

View commented answer

Typical examples: approving a commercial exception for a client and responding to a formal complaint. Both require context, empathy, and responsibility that you can’t transfer. The agent can prepare the information, but the decision and signature are yours.

6

The cycle: error becomes a rule

Executes Observes Adjusts a rule Evolves
Each turn of the cycle transforms an error into a rule, and the sum of the turns is what justifies moving up a level.

What it is

The closing of the seven principles is a cycle: the agent executes, the report and the failure log show what happened, you analyze, adjust one rule in the Agent Sheet, and the agent goes back to execute with the new rule. Every well-documented error becomes one more line in the rules, in the prohibitions, in the triggers, or in the official source. Over time, the sheet becomes more accurate, and the agent can move up a level with evidence.

Why learn

Without the cycle, the same error comes back every month and the solution seems to be swapping tools or rewriting everything. With the cycle, each failure makes the system a little stronger, and the supervision effort decreases. That’s how an agent starts at N1 and, months later, runs a whole process safely.

Key concepts

Executesthe agent works within the agreed level.
Observesreport and failure log show what happened.
Adjustone rule at a time, with the smallest possible correction.
Improvewith fewer mistakes, more evidence—maybe one level up.
COMMENTED EXAMPLE · 3.2.6
Goal: close the loop on an error, from the log to the updated instruction.

1. ERROR (log): <20/09 · included a client who had already sent everything in the pending list>
2. CAUSE: <looked at the old folder, not the folder for the month>
3. SMALLEST CORRECTION: <check only the folder with the current month in its name>
4. NEW RULE ON THE SHEET (Source section): “<Only the folder for the current month counts. Folders from previous months don’t count.>”
5. TEST: <put one document only in the old folder and see if it’s marked as pending>
6. RESULT: <passed in dd/mm> · next review: <dd/mm>

How to check: the same error does not appear in the failure log in the next 4 weeks.

✓ Do this

✓ Close each error with a new rule and a test that proves the rule works.

✗ Avoid this mistake

✗ Fix the result manually and move on without changing anything in the sheet.

Practice before you reveal

Take a recent error from any agent or assistant you use and go through the 6 steps of the example.

View commented answer

The hardest step is usually the smallest correction: the temptation is to change too much. If the correction fits in one sentence and can be tested, it’s the right size. If the error doesn’t reappear in 4 weeks, the cycle worked—and that counts as evidence for the level.

CHECK WITHOUT LOCKING

Check your understanding

The agent mispriced a service in yesterday’s report. What is the correct supervision response?

What you take from this module

Build the report, the failure log, the triggers, and the supervision ritual for an agent—and turn every error into a rule.

  • The report you want to read.
  • Failure log: one line per error.
  • Triggers: when it stops and calls you.
  • The ritual: 2 minutes per day, 10 per week, 1 review per month.
  • Supervision is human responsibility.
  • The cycle: error becomes a rule.

Next action: apply what you learned to your agent sheet and note what still needs review.

Reference for this module: project 7PA materials and course sources and editorial criteria.