Question 11 · 🛡️ Security testing
Do you write different instructions for smaller models than for frontier models?

Many people switch models to save money or gain speed and are surprised when the results get worse. In most cases, the problem isn’t the model; it’s the instruction, which was written for a model that understood more on its own.
Frontier models are better at understanding what you mean, so you can leave more things implied.
With smaller models, it helps to give clearer goals, more context, and more guidance when they get stuck.
For security work, everything starts with the client's specific question: Are customer records exposed? Can a payment safeguard be bypassed? Or do they need a general assessment? These are different engagements, and the request and scope should reflect that.
What changes from one model to another
An advanced model fills in gaps with common sense. If you ask it to review a contract, it assumes you want it to look at deadlines, penalties, and unusual clauses. A smaller model does exactly what’s written and stops there.
That’s why the same instruction (the prompt, or text you send to the AI) can produce a great result with one model and a shallow result with another. The smaller model isn’t dumb; it just needs more guidance.
Three ways to help smaller models
First, a clear goal: say what success looks like in a way you can verify. Instead of asking it to analyze the website, ask it to list pages that show customer data without requiring a login.
Second, context (the background information the AI needs to understand the situation): what system it is, who it is for, and what has already been tested. Third, direction when it gets stuck: if it repeats the same attempt twice, tell it to try a different approach and suggest one.
In security, scope comes first
Scope is the agreed limit of the work: what can be tested, how, and how far to go. It comes from the client’s question, not from a desire to test everything.
Checking whether customer data is exposed is one investigation. Checking whether the payment lock can be bypassed is another. A general review is a third. Each calls for different instructions, tools, and reports, and a smaller model can hit the target only if the scope is written in the instructions.
When a smaller model makes sense
Smaller models are usually cheaper and faster. They’re useful for repetitive, well-defined tasks where you can describe the steps. For open-ended tasks, where even you don’t know the path yet, an advanced model tends to get more done. Test both with the same task before deciding.
Frontier model
- Short prompt
- Understands what's implied
Smaller model
- Clear goal
- More context
- Guidance when it gets stuck
- Write the client’s question in one sentence before giving any instructions.
- Turn that question into a scope: what is allowed, what isn’t, and how far to go.
- Define what counts as a result in a way that can be checked.
- Add the context the model can’t guess.
- Run the same task with the smaller model and the advanced one, then compare.
- Save the instruction that worked so you can reuse it.
- Using the same short prompt with a smaller model that you use with an advanced model: it won’t fill in what’s missing.
- Starting a test without a written scope, which mixes different tasks and confuses the report.
- Letting the agent keep trying the same approach, wasting time and money without making progress.
- Frontier model: you can imply more.
- Smaller model: clear goal, context, and guidance.
- The client's question defines the scope.