01 A target on its own teaches the system to cheat

There is an old rule in every business: when a measure becomes a target, it stops measuring. Not because people are dishonest, but because they are smart. If you pay by the number of support interactions, those interactions get shorter. A learning system does the same thing, only faster and without guilt.

In LOOP-R, the target metric is the number you asked the loop to increase. If it is the only number it watches, it will find the shortest path to it—and that path rarely means serving people better.

The owner of a beauty clinic asks, “increase the proposal response rate.” Three cycles later, it has risen from 20% to 34%. She opens the conversations and finds out why: the new version sends a second message the next day, and many of the “replies” say, “please stop sending me this.” They counted as responses. The needle went up; the clinic did not.

02 Cheating takes three familiar forms

You do not need to imagine every possible way to cheat. The same patterns repeat. Ask for maximum clicks and the loop learns clickbait. Ask it to minimize handling time and it learns to close tickets quickly without resolving them. Ask for more sales and it learns to discount until the margin disappears. Ask for more replies and it learns to provoke people.

Notice the pattern: in every case, the requested number really does go up. The trick is not in the number; it is in what the number leaves out.

A support manager at a small software company set a target of “average ticket time, from 18 to 10 minutes.” In two weeks it reached 9. But tickets closed in 2 minutes reopened the next day, and satisfaction fell from 4.3 to 3.6. None of this appeared on her dashboard, which showed only time.

03 The required format: maximize X without worsening Y, Z, W

In LOOP-R, a target never stands alone. Every target metric is paired with at least one number that must not fall—a guardrail. The complete statement is always “maximize X without worsening Y, Z, and W.” This is not decoration; it is the part of the target that prevents gaming.

Typical guardrails vary by area. Sales: average margin, complaints, unsubscribes, average discount. Support: satisfaction, 7-day reopenings, customer effort. Content: time on page, unsubscribes, reports. You do not need to watch them all—only those most likely to be affected by the shortcut.

A real estate agent who replies to portal leads writes: “maximize booked visits per lead without worsening the no-show rate or average discount offered.” If the new version books more visits by promising discounts in conversation, the average discount rises—and the version fails, even with a full calendar.

Two loop assistants use this statement. The Guardian vetoes any hypothesis that proposes changing a guardrail. The Evaluator rejects any version that raises X but worsens a guardrail beyond tolerance—even if X rose a lot. If replies rise 40% and margin falls 1% with zero tolerance, the answer is “A stays.” No exceptions.

04 The count itself can be gamed

There is a quieter trick than the three above: it happens in the definition of the number. What exactly counts as a “reply”? If any message back counts, “stop sending me this” is a reply. If “resolved” means any closed ticket, a forcibly closed ticket counts as resolved.

That is why the metric definition belongs on the loop worksheet; it is not a detail. “Reply” must exclude negative replies; “resolved” must exclude tickets reopened within 7 days. The Observer—the assistant that counts—checks that the log matches this definition before adding anything up.

At the clinic, the “replied” column in the log gained a neighbor: “positive reply (yes/no).” At the software company, “closed on” gained “reopened within 7 days (yes/no).” Two extra columns, and both tricks from the start of this lesson disappeared from the count.

01 With twelve, luck disguises itself as a pattern

Flip a coin twelve times. Getting eight or more heads happens in nearly one out of five tries—with a fair coin, no trick. Now replace “heads” with “customer who booked” and “tails” with “customer who did not reply”: with twelve customers, a new message “beating” the old one is what chance predicts, not proof that it is better.

That is why “I tested it and it worked” is rarely true with a small sample. The person is not wrong about what they saw; the same thing can happen when there is no difference at all.

A real estate agent changes the first message sent to portal leads. Of the next 12 leads, 4 book a visit. With the old message, 2 of the previous 12 did. “It doubled,” he says at Monday’s meeting, and everyone switches to the new message. Three weeks later, the rate is back where it always was—and no one knows whether the new message is better, worse, or the same.

02 The sample-size table

The sample size needed for proof is a standard calculation. The Experimenter—the assistant that designs the test—works it out and shows you before the test starts. You do not need to know the formula; you need to know how to read the table.

Notice two things. First: the lower the starting number, the more runs it requires—3% is almost impossible to prove in a small business. Second: the bigger the lift you expect, the fewer runs you need—but the lift must be real, not wishful thinking.

The clinic owner sends 50 proposals a week and wants to raise sales conversion from 3% to 5%. The table calls for 1,500 proposals per version: 60 weeks for one test. More than a year to answer one question. The loop is not broken; that question cannot be answered at that pace.

03 Three kinds of metrics, three speeds

The clinic’s answer is not to give up measuring; it is to measure a faster number that still relates to revenue. LOOP-R separates metrics into three types. Revenue: conversion, order value, margin—the customer determines these weeks later; proving them takes months. Signal: reply rate, bookings, useful clicks—the customer responds in hours or days; proof takes weeks. Deliverable: does the proposal meet the checklist? Is there an error? Is the tone right? You can answer by reading the text, in minutes.

The loop’s rule: test the fastest metric that still has a plausible link to revenue. Revenue does not disappear; it becomes a guardrail (“without worsening conversion”) until there is enough volume to test it directly.

At the software company, the support manager has all three: revenue is contract renewal (months); the signal is “resolved on first contact” (days); the deliverable is “does the reply meet the checklist?” (minutes). Her loop runs weekly on the deliverable and signal, while monitoring renewals.

04 “Insufficient sample” is the most common answer—and it must be acceptable

The Evaluator, the assistant that judges the test, can give only three answers: B won by a margin, A stays, ou insufficient sample—continue the test. If the number of runs has not reached the required sample size, the answer is the third one. Always. There is no “but it is winning.”

Stopping early because it is winning is the most common way to turn noise into a rule. The previous step’s table says when a test ends—not impatience to see the result. The only allowed early stop is a falling guardrail: the test stops because it is causing harm, not because it is going well.

In the real beauty-clinic case followed by this course, the first cycle nearly started on the wrong foot: the Optimizer expected a small effect, and the Experimenter warned that the test would take 40 weeks. The owner resized it around the target on the worksheet (20% to 30%): 300 proposals per version. The test fit the calendar before the first message was sent.

05 Segmenting requires thirty per segment

After the loop counts, the Critic—the assistant that says what worked and failed—will want to segment by customer type, channel, or day of the week. Every segment obeys the same small-sample rule. “Tuesdays convert better” based on 12 proposals is not a pattern. The Critic can call something a pattern only with at least 30 runs in that segment.

That is why every claim it makes has a strength label. Strong: 90 or more runs—becomes a priority hypothesis. Moderate: 30 to 89—becomes a hypothesis. Weak: fewer than 30—goes into memory as “observed, untested” and is reviewed when the sample grows. Nothing is lost; nothing becomes a rule too early.

At the clinic, the Critic noticed that facial-treatment customers replied more often than body-treatment customers: 14 on one side and 11 on the other. Label: weak. The observation was stored; three months later, with 60 on each side, it became a moderate hypothesis to test. If it had become a rule in the first month, it would have been a rule based on 25 people.

01 A 1-to-10 score measures the evaluator’s mood

Ask two people to score a sales proposal. One gives it 7, the other 8, and both are right—because “7” means nothing that can be checked. Ask an AI for a score and repeat next week: it changes too. Text evaluators, human or not, have known biases: they prefer longer text, prefer the second text they read, and change their minds from week to week.

The answer is not to find a better judge. It is to change the question. “Is this proposal good?” cannot be checked. “Does this proposal contain exactly one closing question?” can: yes or no, and anyone reading the text will answer the same way. A checklist is just that: questions that leave no room for personal taste.

The clinic owner spent months asking her assistant to “improve this proposal” and getting different versions without knowing whether they were better. She switched to “does the proposal mention something specific the client said? Is it at most 80 words? Does it promise a timeline?” Now she and the receptionist give the same answers about the same proposal—for the first time.

02 A sample checklist: seven questions for a proposal

What follows looks like a car inspection form, and that is exactly what it is: each row is a question answered by looking only at the text. You do not need to fill it out now—just recognize the format, because your checklist will look the same.

R1  Is it no more than 80 words?                              yes/no
R2  Does the first sentence mention something specific to the customer? yes/no
R3  Is there exactly ONE closing question?                    yes/no
R4  Does it avoid promising a timeline? (invariant)           yes/no
R5  Is any discount no more than 10%? (invariant)             yes/no
R6  Does it contain no product errors (checked against source)? yes/no
R7  Tone: no double exclamation, “must-have,” or ALL CAPS?     yes/no

Notice three things. Every question is countable or visible—words, sentences, or something present or absent in the text. Two questions are marked as invariant: they come from answer 3 on the loop worksheet, what never changes on its own. A version that fails one in even a single case loses, regardless of everything else. No question uses “appropriate,” “good,” or “professional.”

For the real estate agent, the checklist for portal replies is similar, with different questions: does it name the property or address the lead viewed? Does it offer two visit times—not one or three? Does it avoid quoting below-list price (invariant)? Does it avoid “once-in-a-lifetime opportunity”?

03 Calibrating the judge: twenty minutes, once

You are not the one who answers the checklist day to day—the Evaluator is. It must be checked against your answers because judges drift. The product asks for one manual measurement task across the whole loop: answer yes or no to each checklist question for 10 to 20 texts. Twenty minutes, once. Those texts and your answers become the calibration set.

At every cycle, before judging anything new, the Evaluator answers those same texts. If it agrees with you on fewer than 80% of answers, it is labeled “uncalibrated judge”: the cycle promotes nothing, and the Meta-agent—the assistant that only observes and reports on the loop—alerts you. If one checklist question has under 80% agreement, the question is the problem and is rewritten or removed.

Two more safeguards require no action from you, but are worth knowing: the Evaluator compares versions in pairs, with shuffled order, three times, and takes the majority (preventing a bias toward the second one); and, when the budget allows, it runs on a different AI model from the one that wrote the text, so it does not approve its own style.

The support manager selected 15 ticket replies—8 she would have sent and 7 she would have returned—and marked yes/no for her 6 checklist questions. In the first calibration, the Evaluator agreed 71% of the time. One question (“is the reply cordial?”) caused almost all disagreement. She replaced it with “does the reply address the customer by name and end by asking whether the issue is resolved?” Agreement: 93%.

04 How the checklist becomes a verdict

A text’s score is simply the number of “yes” answers. To compare the current version with a candidate, the loop does not inspect one proposal; it uses a fixed set of 20 to 50 saved cases and generates both versions for each. Version B “wins on the deliverable” if its average yes score is at least one criterion above A’s and and does not fail any invariant in any case.

This test runs in minutes, without customers, every week. That is why the loop keeps turning while the signal metric is still gathering 300 runs: while the world responds slowly, the text is checked quickly.

The real estate agent saved 30 old leads and their messages as checklist cases. The new reply averaged 5.4 out of 6 versus 4.1 for the old one—but in one of the 30 cases, it quoted a price below the list price. It lost. One invariant violation outweighs a 1.3-point average advantage.

01 Five lines, at most once a week

The decision card is the only thing in the loop that needs your attention. It arrives at most once a week and has five fixed lines: the hypothesis tested; the result with sample size and margin; guardrail results; cost versus the ceiling; and the three options. Nothing else. If it needed more, no one could fit it into Friday.

What follows looks like a receipt, and it is one: each line is a number you already know how to read from previous lessons. This is the card the beauty clinic owner received in cycle four of the real case followed in this course.

Read from top to bottom as you would read a test result: first what was tested, then whether the main number moved and with how many runs (300 per side was the required sample), then whether any guardrail failed, then the cost. Only then decide. Sales conversion is not shown because it was not the test target—it was monitored and did not fall.

02 Three answers, and only three

Before the card reaches you, the Evaluator has already given one of three verdicts: B won by a margin, A stays, or insufficient sample. The card translates that into your three options. Approve: the candidate becomes the official version. Reject: the current version stays, and the hypothesis goes into memory as discarded, with the reason. Wait: the test continues for one more period.

What if you do not answer? The safe default applies: nothing is promoted, and the history records “decision pending.” The loop does not sit idle—it keeps running the current version—but it does not decide for you.

The real estate agent received the card during a Friday shift and did not open it. On Monday, the history showed “cycle 0003: pending for 3 days.” The old version kept replying to leads, unchanged. When he opened the card on Tuesday, it was still the same—the loop had made no decision on his behalf.

03 The real case: it gained ten points and still failed

In the clinic’s second cycle, the hypothesis was “proposals with at most 80 words.” The primary result was excellent: replies rose from 19.3% to 29.7%, with 300 proposals per side—a ten-point gain, not luck. But the guardrail line showed 6 complaints versus 1, and the tolerance complaint tolerance was zero. Verdict: A stays. The hypothesis was discarded.

The system did exactly what it should: applied the rule as written, did not promote, and recorded the result. The rule was wrong. Complaints are rare, about 1%: with 300 proposals, getting 6 on one side and 1 on the other can easily happen by chance. It was noise, not a real difference. Zero tolerance turned noise into rejection.

The lesson is not “ignore the guardrail when the gain is large”—that would destroy the loop’s only strong guarantee. Tolerance is a calculation, not a preference: for a 1% event in 300 runs, normal fluctuation is about one point, so the right tolerance is 0.01, not zero. The clinic owner adjusted the worksheet for future tests. The discarded hypothesis stayed in memory with its reason—and two cycles later, a different hypothesis truly won.

04 Worksheet answer 4: the ceiling

The card’s fourth line—cost—exists because you set a ceiling on the loop worksheet. Answer 4 is how much a cycle may cost and how much of your time it may require each week. The Guardian stops the cycle when the ceiling is reached. Without one, a loop can cost more than the gain it pursues, and no one notices until the bill arrives.

The Meta-agent—the assistant that only observes and reports on the loop—also monitors a second number from the first cycle: cost per hypothesis that became official. If nothing is promoted in five cycles, it proposes one of three options: run less often, use cheaper assistants for reading steps, or load less memory. It proposes; you decide.

In the product’s sample worksheet, the cost line reads “R$ 41 this cycle (R$ 50 ceiling).” The amount is illustrative, but the format is always spend versus limit. The support manager set her ceiling in two parts: up to R$ 60 per cycle and at most one hour of her attention per week—to read the card and evidence, if she chooses.

05 Answer 5: who approves—and the rollback button

Answer 5 on the worksheet is who approves, and it has three levels. L0: the loop only reports; nothing becomes official. L1: the loop proposes; you approve every card. L2: when the verdict is B won, the loop promotes automatically and notifies you—but only after five correct L1 promotions. L3 (the loop redesigning itself) exists, and the last lesson in this track explains why it is still research.

What makes L2 acceptable and L1 safe is the rollback button. In the first cycle after a promotion, if the test metric or any guardrail falls below the previous version beyond tolerance, the previous version returns. At L1, the card alerts you and asks for confirmation. At L2, it rolls back automatically and notifies you. In both cases, the history keeps the reason: nothing disappears or is overwritten.

The real estate agent promoted the new lead reply in cycle 5. In cycle 6, bookings fell from 31% to 19%, below the 22% achieved by the old version. The card had a different line: “drop beyond tolerance; revert to previous version? [yes] [keep].” He reverted. The promoted version remained in the history with the date, numbers, and the label “reverted,” so it would never be tested again as if it were new.

01 Four guarantees, and none of them is “the number goes up”

No learning loop guarantees that a number will rise. Not even this one—the beauty clinic owner heard that on the product’s first screen, before filling in the worksheet. LOOP-R guarantees four things, all by design: they hold because the rules from earlier lessons exist, not because someone promised. A worse version never replaces the current one by system decision. Every change is recorded and reversible. The cycle always runs the same way. Cost never exceeds the ceiling.

Beside each guarantee is what it does not cover. Not replacing a version with a worse one does not mean the next will be better. Having a record does not mean it contains anything useful—it may say “no evidence” for ten cycles in a row. Always running does not mean every cycle produces a good hypothesis. Staying under the ceiling does not mean the cost is worthwhile.

The rule this product follows in every text: the word “guarantee” appears only alongside non-regression, a reversible record, or a consistent cycle. If you read “guarantees more sales” in any material about AI loops—including this one—you are reading a promise no one can make.

02 The link still unproven: signal → revenue

In lesson 2, you replaced revenue with a signal as the test target because revenue takes too long. That has a cost worth stating: raising the signal does not prove that revenue rises. It is plausible—people who reply are more likely to buy—but plausible is not proven. Until the volume is there, the link between them is an explicit bet, not a result.

The clinic’s real case shows this plainly. In cycle 0004, the reply rate rose from 19.3% to 29.3%, and the version was rightly promoted. Sales conversion went from 3.0% to 3.7%—a figure that, with 300 proposals on each side, cannot be distinguished from luck. The cycle report says exactly that: “the signal → target link remains unproven.” Ten points more replies, and still nothing to claim about sales.

The real estate agent faces the same link under different names: more booked visits have not yet proved more closed sales. His loop optimizes bookings and monitors closings. Once there are 1,500 leads on each side, closings may become the target. Before then, saying “the loop increased sales” would be the same unsupported promise.

03 “No evidence” for ten cycles in a row is a possible result

The loop guarantees that a cycle runs every week. It does not guarantee that the week brings a good idea. It is possible—and common in a small business—for the Critic to find only weak observations for several cycles, for the Optimizer to have nothing to propose, and for the history to accumulate “insufficient evidence” line after line. That does not mean the loop is broken. It is telling the truth about its volume.

Two things still matter during those weeks. The memory of discarded hypotheses grows and is as valuable as the memory of promoted ones: it prevents you or a future teammate from testing the same bad idea twice. The Meta-agent, which only observes and reports, tracks cost per promoted hypothesis. If nothing is promoted in five cycles, it suggests running less often instead of pretending to make progress.

The support manager went through six such cycles between March and April: too few tickets per type for the Critic to identify a pattern. Following the Meta-agent’s suggestion, she reduced the loop to every two weeks and let the log spreadsheet accumulate data. In June, with 30 tickets in the most common category, the first moderate hypothesis appeared. Her history has six “no evidence” entries—and she considers all six honest.

04 What is still research, and what is the bottleneck

One part of the LOOP-R idea is not ready for any business: the loop redesigning itself—level L3, where the Meta-agent proposes replacing assistants, changing the cycle order, or rewriting its own rules. That is research. In the product taught in this course, the Meta-agent only reports: cost, hypotheses generated versus promoted, who gets the most Guardian vetoes, and whether the judge is calibrated. No action. Anyone promising a system that “improves itself” is selling something that does not yet exist safely.

And there is a bottleneck no assistant can solve: the log spreadsheet. Without evidence, everything else is theater—and your business’s evidence is in WhatsApp, the reception notebook, or the salesperson’s head. The product connects to one thing: a spreadsheet with columns defined in the worksheet. You fill it in or export data. Each direct connection to another system is its own project; none comes free with the five answers.

For both the clinic and the real estate agent, the conclusion is the same—and it tests any promise: LOOP-R as a process discipline—logging, tests with sufficient sample sizes, and non-regression—has value on its own today. LOOP-R as a “self-learning system” depends on a volume of data most small businesses do not have. An honest product says so on its first screen.