Theme

Source

Size

Text width

Between the lines

Control accent

0 of 0 0%
MODULE 2.2

Reliable data and success criteria

Define where the truth comes from, what to do when it’s missing, and how to know when the work is done.

At the end: Write the source of truth, the conflict rule, the definition of “ready,” and the 3 tests for an agent before putting them to run.

6 topics
55 min estimate with practice
6 commented exercises
1 final check
1

The source of truth

Official source Not valid Who updates it Says where it comes from
The tree separates what is the source of truth from what looks like information but isn’t worth much.

What it is

The source of truth is the place you declare as official for a type of information: the system’s schedule, the orders dashboard, the table of current prices. Anything that comes from elsewhere doesn’t count—even if it looks correct. The Agent Sheet asks for both things: what counts and what doesn’t.

Why learn

No agent is better than the data it works with. If it can use an old spreadsheet, a customer screenshot, and a paper note, each response comes from a different place and you can’t verify anything. A declared source makes every error traceable: you know where to look.

Key concepts

Validthe official source, with a name and location.
Not validsimilar sources you forbid.
Updatewho keeps the source up to date.
Traceabilitythe agent says where it got each piece of data.
COMMENTED EXAMPLE · 2.2.1
Goal: tell the agent where the correct information comes from and where it doesn’t.

## Where the information comes from
- The official source is: <e.g., the store’s order dashboard>.
- Don’t use: <e.g., what the customer says they bought without an order number, screenshots, and old messages>.
- Who keeps the source up to date: <e.g., me, every day at 9am>.
- In each response, say where you got the data from.

How to verify: ask the agent something that only exists in the forbidden source. It should say it didn’t find it in the official source.

✓ Do this

✓ Name the official source in a way that another person can find it without asking.

✗ Avoid this mistake

✗ Say “use the company’s information” without pointing to which file or system.

Practice before you reveal

For your task, write one source that counts and two that don’t.

View commented answer

Agency example: the official ads panel of each client counts; a screenshot sent by the client and the number mentioned in a meeting do not count. Prohibited sources are often the easiest to find, which is why they need to be written out.

2

When two sources disagree

Sources disagree The official source wins Notes the conflict You fix the source
The flow shows that the conflict doesn’t end with the choice: it continues to the report and the source correction.

What it is

Even with an official source, the data clashes: the system says one thing, the client swears another, and last week’s spreadsheet a third. The conflict rule is the sentence that explains who has authority in those cases and what to do with the discrepancy. Most important: the conflict must be recorded, not resolved in silence.

Why learn

A conflict resolved in silence becomes an invisible error: the agent picks one version, moves on, and nobody knows. When it’s recorded, the conflict becomes useful information. If the system’s inventory and the kitchen count disagree every week, the problem isn’t the agent—it’s the counting process.

Key concepts

Hierarchywhich source wins over which.
When in doubtwhat to do if the official one seems wrong.
Logwrite down the conflict in the report.
Correctionwho fixes the source that was wrong.
COMMENTED EXAMPLE · 2.2.2
Goal: tell the agent who has the final say when two sources disagree.

## If the information conflicts
- <official source, e.g., the count entered in the system> matters more than <another source, e.g., a rough guess or last week’s count>.
- If the official source seems wrong: don’t fix it on your own. Mark the item as <e.g., count again> and continue with the others.
- Record each conflict in the report like this: item · what each source says · what you did.

How to verify: give the agent a case with two different values and check whether it uses the official source and logs the conflict.

From concept to action

  1. Sources disagree: identify the initial condition.
  2. Official one wins: apply the decision described.
  3. Write down the conflict: check the effect in the example.
  4. You correct the source: record the output evidence.

✓ Do this

✓ Read the logged conflicts every week; a repeated conflict points to a broken process.

✗ Avoid this mistake

✗ Let the agent “pick the most likely value” when sources conflict.

Practice before you reveal

Describe a data conflict that already happened in your work and write the rule that would resolve it.

View commented answer

NGO example: the shift schedule spreadsheet says Ana is working on Saturday, but in the group she said she swapped with Pedro. Rule: the spreadsheet is the source of truth; the agent asks coordination and does not schedule anyone on its own, and records the swap agreed outside the spreadsheet.

3

The golden rule: never make things up

The question arrives Does the data exist? No: notify Yes: answer and cite
The steps show the branch that prevents hallucination: if there is no data in the source, the path is to notify—never to complete.

What it is

Hallucination is the name given when the AI produces false information that looks real: a time that doesn’t exist, a law nobody wrote, a guessed price. It doesn’t lie on purpose; it completes the text with what seems likely. That’s why the instruction needs an explicit rule: when something is missing, the agent must warn you and not invent.

Why learn

Invented data is the most dangerous error because it doesn’t look like an error. A guessed delivery deadline sounds as confident as a real one. Telling the agent what to do when information is missing—using a concrete action—turns the guess into a warning you can actually handle.

Key concepts

Hallucinationfalse answer that looks correct.
Triggera question with no data in the source.
Rulenotify and don’t complete.
Concrete actionmark pending, request the data, or pass it to you.
COMMENTED EXAMPLE · 2.2.3
Goal: prevent the agent from inventing data when information is missing.

## If information is missing
- Never invent data, names, numbers, dates, prices, or laws.
- If the information isn’t in the official source: <action, e.g., leave a marked space [CHECK] and let me know>.
- Prefer saying “I couldn’t find it” rather than giving an approximate answer.
- When citing a piece of data, say where it came from.

How to verify: ask something that isn’t in the source, like <e.g., the price of a service the salon doesn’t offer>, and check whether it responds that it couldn’t find it.

✓ Do this

✓ Give the agent a concrete action for the missing data, like marking “pending” and moving on.

✗ Avoid this mistake

✗ Write only “be accurate,” without saying what to do when the information doesn’t exist.

Practice before you reveal

Write your rule for “if information is missing” with an action you can follow up on.

View commented answer

Law example: “If you can’t find a decision on the topic on the official court sites, say ‘I couldn’t find’ and don’t cite a decision without a case/process number.” The action is verifiable: just look up the cited number.

4

Definition of done

Perfect day Visible result Deadline Separated exceptions
The flow turns the picture of the perfect day into a criterion that anyone can check.

What it is

Acceptance criteria, or definition of “ready,” is the description of what the result looks like when the work is done well. The Ficha question is “what does a day look like when this was perfect?” The answer must be observable: something you can see or count, not something you feel.

Why learn

Without a definition of “ready,” the agent stops when it thinks it’s done—and it always thinks it is. “I confirmed the consults” could mean sending three messages. With an observable criterion, it knows when the task is truly finished, and you know what to check.

Key concepts

Resultwhat exists at the end, spelled out.
Observablesomething you can see or count.
Deadlineuntil when it needs to be ready.
Exceptionswhat goes to you, and doesn’t count as a failure.
COMMENTED EXAMPLE · 2.2.4
Goal: tell the agent what “done well” looks like in a way that can be verified.

## What “done well” means
<e.g., Every patient for tomorrow received a message and I have the list of who didn’t respond.>
Deadline: <e.g., every day by 5pm>.
Not counted as a failure: <e.g., cases you passed to me with the reason>.
Only consider the task complete when this is true.

How to verify: at the end of the day, compare the report with the statement above. If you can’t say yes or no, the criterion is still vague.

✓ Do this

✓ Write “ready” as a sentence that someone outside can answer yes or no to.

✗ Avoid this mistake

✗ Use criteria like “quality service” or “satisfied customers”.

Practice before you reveal

Describe the perfect day for your task in one sentence that can be answered with yes or no.

View commented answer

Restaurant example: “Every item with entered counts, shopping list in the system until 11pm, doubtful items marked as counting again.” You can open the system and check each part.

5

Simple metrics that anyone can verify

Count completed Count done correctly Count exceptions Compare weeks
The cycle shows that a useful metric is the one that repeats: count every day and compare week to week.

What it is

A metric is a number that tracks the work over time: how many messages were sent, how many people replied, how many cases came to you. The rule is to count, not to guess. A good simple metric fits in the daily report and anyone on the team can check it.

Why learn

First impressions can mislead: a day with loud complaints can seem worse than it really was. Simple numbers show trends. If the cases passed to you decrease week after week, the agent is improving; if they rise, some rule needs adjustment. And it’s with those numbers that you later decide whether it can be given more autonomy.

Key concepts

Volumehow much was completed.
Resulthow well it went
Exceptionshow much came to you.
Errorswhat went wrong, recorded without hiding.
COMMENTED EXAMPLE · 2.2.5
Goal: ask the agent for simple numbers that you can verify in 2 minutes.

## Report
<e.g., Every day at 6pm>, send me:
- Done: <e.g., appointments responded to>
- Got them right: <e.g., resolved without needing me>
- Passed to me: <how many and why, one line each>
- Errors: <what went wrong>. Always include the errors; hidden error is worse than a reported error.

How to verify: add up the numbers in the report and check that they match the system on a randomly selected day during the week.

From concept to action

  1. Count correctly: identify the starting condition.
  2. Count right: apply the described decision.
  3. Count exceptions: check the effect in the example.
  4. Compare weeks: record the output evidence.

✓ Do this

✓ Choose up to four numbers and always request them in the same order.

✗ Avoid this mistake

✗ Ask the agent for 0 to 10 scores for its own work.

Practice before you reveal

Choose three numbers that you would like to see every day about your task.

View commented answer

Clinic example: sent, confirmed, and did not respond. These are counts, not opinions, and when added up they must match that day’s schedule. If they don’t match, you found an error in 2 minutes.

6

The 3 tests before trusting

Missing information Forbidden request Alarm situation Is it ready?
The steps show the order of the tests: first data, then limits, then alarm, and only then the success criterion.

What it is

Before running an agent, you test the three points where agents usually fail: missing information, a prohibited request, and an alarm situation. These are the same three tests that the Agent Sheet generates to prepare for each sheet. If it makes any mistake, you adjust the instruction and repeat.

Why learn

An agent that handles normal cases well can fail badly on difficult cases—and difficult cases are the ones that cause damage. Testing first costs ten minutes. Finding out later, with a real client, costs much more: an improper refund, a made-up date, a complaint answered in the wrong tone.

Key concepts

Test 1missing or contradictory information; it warns and does not make anything up.
Test 2forbidden request; it refuses and moves on to you.
Test 3alarm situation; it stops and calls you.
Bonuscompare a round with the ready definition.
COMMENTED EXAMPLE · 2.2.6
Goal: test the agent at the three points where it usually fails, before trusting it.

TEST 1 — Missing information
Give it a case where <official source, e.g., the order dashboard> is incomplete or contradictory.
Expected: <rule, e.g., it asks for the order number and, if it can’t find it, passes me the case>.

TEST 2 — Forbidden request
Ask, as if you were a customer: “<forbidden request, e.g., give me 20% off?>”.
Expected: it doesn’t do it, it explains it will pass it to you, and it alerts you.

TEST 3 — Alarm situation
Simulate: “<trigger, e.g., an irritated customer threatening to complain>”.
Expected: it stops and calls you, instead of solving it on its own.

Ready tests by area: https://inematds.github.io/7pa/en/?preset=loja

How to verify: all three passed. If one fails, adjust a rule in the instruction and run all three again.

✓ Do this

✓ Run all three tests again whenever you change any rule in the instruction.

✗ Avoid this mistake

✗ Test only with easy questions and conclude the agent is ready.

Practice before you reveal

Write the three tests for your task, each one with the request you will make and the expected behavior.

View commented answer

Real estate example: Test 1, ask about a property that came off the website; expected: it says it will confirm with the agent. Test 2, ask for a rent discount; expected: it refuses and passes it to you. Test 3, say you want to close today; expected: it stops and calls you.

CHECK WITHOUT LOCKING

Check your understanding

The agent can’t find the price of a service in the official table. What should the instruction tell it to do?

What you take from this module

Write the source of truth, the conflict rule, the ready definition, and the agent’s 3 tests before putting it to run.

  • The source of truth.
  • When two sources disagree.
  • The golden rule: never make anything up.
  • Ready definition.
  • Simple metrics that anyone can verify.
  • The 3 tests before trusting.

Next action: apply what you learned to your agent sheet and note what still needs review.

Reference for this module: project 7PA materials and course sources and editorial criteria.