Does the agent get “smarter” as it tests? And is trying to change a price the same as succeeding?

Security reports from AI agents often come packed with findings. The question that separates serious work from rushed work is simple: was this attempted or proven?

Imagine an authorized test on a small online store: the agent tries to buy a product by changing the price along the way.

There's a big difference between an agent trying something and proving it worked. Seeing an attempt to change a price doesn't prove that the server (the store's computer that actually records the order) accepted the change.

During testing, the system gathers information: a failed approach may reveal an address or detail worth investigating in the next attempt. In practice, it's still useful to have a person guiding the agent; write down what worked so you can repeat the test without explaining everything again.

An attempt and proof aren’t the same

The agent may change a product’s price during checkout and show the change in its log. That’s only an attempt. The server (the store’s computer that records the actual order) makes the decision.

It’s proven when the server accepts the change: the order is saved with the altered price, the wrong amount is charged, and the system confirms it. Without that confirmation, the finding is a hypothesis, not a flaw.

How to check whether it worked

Look at the server’s response, not just what the agent sent. Check the final state: the order in the store’s dashboard, the cart total after reloading the page, or the record in the test database.

A good rule is to repeat the test from scratch. If a flaw appears only once and can’t be reproduced, be skeptical. Ask the agent for evidence (a screenshot, full response, or order number) for each claim.

The agent learns during the test

During a session, the agent gathers information. A failed attempt may reveal an internal address, an error message, or a hidden field worth investigating next. In this sense, it gets smarter as the test goes on.

But in general, that learning isn’t saved for next time. If no one records what worked, the next test starts from scratch.

Why someone still needs to be involved

A person guiding the work decides which clues are worth pursuing, cuts off useless paths, and checks whether the proof is real. They also make sure the agent stays within the agreed scope. All of this should happen only in an environment you own or with the system owner’s written permission.

1. Agree on the scope and get written permission before starting.
2. Ask the agent for evidence for each finding, not just a description.
3. Check the result in the system’s final state, such as the saved order.
4. Repeat important findings from scratch to see if they hold up.
5. Record what worked, what failed, and what clues came up.
6. Use those notes as the starting point for the next test.

• Reporting an attempt as a confirmed flaw, which hurts the credibility of the work.
• Trusting only the agent’s log without checking what the server actually saved.
• Not recording what worked, forcing you to explain everything again for each test.

• Attempt ≠ proof.
• Each failure can teach you something for next time.
• A person still needs to guide the process.

• Projectexecucao-longaVerifiable completion criteria and state files that keep the agent on track.
• ProjecthindsightLong-term memory: the agent saves and consolidates what it learns.
• Projectsil-loop-rRecord failures and approve lessons before they become rules.