🎯 Measure what matters
Measuring isn't about filling a dashboard with numbers. It's about choosing the number that proves the problem was solved — and ignore everything else. Most people measure what’s easy to count (clicks, messages, pretty screens) and forget to measure what the intent called for (time saved, pain eliminated, money retained). The right metric comes from intent, not from what the tool gives you for free.
🧭 Measurement answers to intent
In Module 3.1, you defined the result that matters. Measuring is just a matter of going back and asking: "did that result happen?". If the goal was to reduce response time, the metric is response time—not the number of messages sent. A good metric is one that tells you on its own whether it was worth it.
✓ Worth measuring
- ✓Time saved per task
- ✓Support requests resolved without a human
- ✓Errors that stopped happening
- ✓Money or hours saved
✗ Vanity metric
- ✗"The agent responded 900 times"
- ✗"It looked great on mobile"
- ✗"Has lots of features"
- ✗(numbers that don't address the pain)
Key concepts
📐 Define the Right Metric
A good metric has three qualities: it is specific (a number, a unit), honest (you can't pretend it improved) and actionable (if things get worse, you know what to adjust). "Improve customer service" isn't a metric. "Average time until the first response, in minutes" is. The difference is that the second can be recorded, compared, and tracked.
The metric written as a single sentence
Illustrative metric definition outline — one per intent.
💡 Practical tip
Test your metric with the “pretend” question: "can this number improve without the problem improving?". If it can (e.g., “messages sent” goes up just by sending more text), replace it with a metric that can’t be gamed—such as “problems resolved on the first contact.”
Key concepts
⚖️ Before × after
A number by itself doesn’t tell you anything. Is “12 minutes” good or bad? You can only tell by comparing it with the before. That’s why the golden rule of measurement is: capture the baseline BEFORE turning on the solution. If you only measure afterward, you’ve lost the half that proves the result—and you’ll have to guess where you started, which is exactly where the illusion creeps in.
🕒 The measurement timeline
-
1
Measure the before (baseline)
Before using any AI, write down the current numbers: 40 min per service interaction, 30 tickets/day, 8 errors/week.
-
2
Connects the solution
Get the Module 3.4 prototype running for real, with people using it.
-
3
Measure the after (same yardstick)
After a fair period, measure again—the exact same number, in the same way.
-
4
Compare and conclude
The difference between before and after is the result. Without step 1, this calculation is impossible.
✓ Honest comparison
- ✓Same metric at both points in time
- ✓Same way of collecting
- ✓Comparable period (week × week)
- ✓Baseline documented beforehand
✗ Misleading comparison
- ✗Estimate the "before" from memory
- ✗Measure afterward in a different way
- ✗Compare a full day with an empty day
- ✗Never measured the starting point
Key concepts
🪄 The illusion of productivity
Here’s the most dangerous trap in the entire course: without measurement, every deliverable looks like progress. The solution runs, the dashboard flashes, everyone applauds—and no one realizes the problem is still exactly the same size. That’s the illusion of productivity: confusing activity (things happening) with result (the pain decreasing). The only safeguard is to measure before and after.
⚠️ Signs you’re fooling yourself
- ✗You describe the solution by what it does, never by what it changed.
- ✗The achievement belongs to the technology, not to the number that went down (or up).
- ✗No one can say what the “before” was.
- ✗The dashboard shows lots of activity and no results.
Activity
Things happening. Easy to see, easy to celebrate.
- • "The agent responded 900 times"
- • "Ran all week without going down"
- • "It has 12 features"
Outcome
The pain easing. That’s what the company pays for.
- • "Customer service time dropped from 40 to 12 min"
- • "The team gained 6 h per week"
- • "Data entry errors dropped 80%"
💡 Practical tip
Before presenting any solution, do the "outcome sentence test": make sure you can complete "it used to be ___, now it's ___" with two real numbers. If you can't fill in both blanks, you haven't measured yet — you only thought you had.
Key concepts
📊 Read the Numbers
Having the number is half the battle; read the number is the other. Reading well means knowing the difference between a real variation and noise, looking at the trend and not just today’s data point, and being skeptical of a result that looks too good (it’s almost always a measurement error, not a miracle). A dashboard doesn’t think for you — it shows; the architect draws the conclusions.
Illustrative recreation of a results dashboard—not a real screenshot. The bars represent each metric against its target.
🔎 Three questions to ask when reading a number
- Is it a trend, or was it just one day? Look at the trend, not an isolated peak—a good day isn’t a result.
- Is it a signal or noise? Small variations come and go; only a large, stable change counts.
- Too good to be true? A miraculous result is almost always a data collection error—check the source.
Key concepts
🔄 Measurement Drives Evolution
Measuring isn't the end of the line — it's what opens the next round. The number points to exactly where to make a change: if response time drops but satisfaction doesn’t rise, the response is fast and bad; if it handles few things on its own, it lacks context or capability. Without measurement, you adjust in the dark, changing what you “think” needs fixing; with measurement, you adjust with clarity, changing what the numbers show. That’s how measurement becomes the engine of improvement (Module 3.6).
🔧 The number tells you what to adjust
- Fast response but low satisfaction → improve quality (soul, context), not speed.
- Does little on its own → missing context, skill, or tool — go back to Day 2.
- Metric blocked → maybe the bottleneck is somewhere else; remap the problem (Module 3.2).
Key concepts
Self-check (optional): why is measuring the “before” state so important?
📏 Module summary
Next module:
3.6 — Evolve and Present: use what you measured to improve the solution and clearly show its value.