Choose by the task
A model needs to be evaluated by the work it is supposed to do. Data extraction, image analysis, and code review require different capabilities. A model that only receives text won’t start “seeing” images just because the prompt is well written.
Define a small task and the quality criteria before comparing alternatives. Consider errors, time, cost, and the need for review. The cheapest alternative per unit may require so many fixes that the full process ends up costing more.
Why learn
Comparison no longer depends on generic rankings. You evaluate the cost and quality of the delivery you truly need to produce.
✓ Do
Load credentials outside versioned files.
✗ Avoid
Copy a key into a public example.
Key concepts
Separate account, model, and interface
The interface is the program you interact with. The provider offers the service, and the model carries out the task. A product subscription and using an API can have different charges and limits. Check that on the account you’re using.
Integrations with alternative providers, such as DeepSeek or MIMO, depend on the compatibility supported by the tool. Don’t assume that swapping only a name is enough. Endpoints, authentication, tool resources, and limits must be compatible.
Why learn
Understanding the layers helps you locate failures and prevents attributing an authentication, network, or interface configuration issue to a model.
A way to apply
- Record the current configuration and the task criteria.
- Test access and capabilities with a small input.
- Compare results and keep a documented rollback.
Key concepts
Create profiles without secrets
A profile describes settings, but it shouldn’t serve as a place to store credentials. Use environment variable names to indicate where the key will be loaded from. An example file needs to work as documentation, without containing real values.
Before changing an existing configuration, save a copy and confirm which file the installed version uses. Some tools allow profiles or options per run; others require specific files. Check the current help instead of renaming files by trial and error.
Why learn
Separating configuration and credential makes it easier to share examples without exposing access. Preserving the original configuration keeps the test reversible.
{
"nome": "perfil-de-exemplo",
"provedor": "preencher-conforme-documentacao",
"modelo": "identificador-validado",
"chave_env": "PROVEDOR_API_KEY",
"tarefa_teste": "extrair-pedidos"
}
Este é um esquema didático, não um arquivo de configuração de uma CLI específica.
Key concepts
Run a connectivity and function test
Start with a small request that doesn’t involve important documents. Confirm that authentication works and that the response uses the expected model. Then test the required capability: structured format, tools, or an image, depending on the task.
A “hello” response confirms part of the path, but doesn’t show that an integration supports all of an agent’s functions. If the tool needs to call operations, test a simple and reversible operation before starting a larger flow.
Why learn
Step-by-step verification separates connection problems from functional limitations. You avoid discovering incompatibilities after you’ve already started a long task.
✓ Do
Compare the same requests and criteria.
✗ Avoid
Choose only based on the lowest advertised price.
Key concepts
Compare using the same yardstick
Use the same set of inputs and record results per criterion. For extraction of requests, count the correct fields, preserved gaps, and format errors. Measure time and usage when the tool provides that data, without estimating numbers you didn’t observe.
Run more than one execution when variability matters. A single excellent response doesn’t demonstrate stability. The decision might be to use one model for drafting and another for a specific step, as long as the added complexity is justified.
Why learn
A comparable matrix shows advantages and limitations without turning preference into evidence. It also lets you repeat the evaluation when the service changes.
| Criterion | Expected evidence |
|---|---|
| Fidelity | It doesn’t make up missing data. |
| Compatibility | Executes the required functions. |
| Rollback | The previous profile can be restored. |
Key concepts
Practice: prepare a comparison
Define a three-request extraction task and compare two fictional profiles. In the exercise, you don’t need to hire services: use the matrix to record clearly labeled simulated results.
Explain why the fastest option may not be the best if you invent deadlines. Then write the procedure for returning to the original profile. The goal is to learn to compare and roll back, not to recommend a provider based on reputation or price from an example.
Why learn
The exercise produces an evaluation protocol that can be applied with real services afterward. It prevents a configuration change from being confused with a demonstrated improvement.
Your exercise
Simulated data: profile A responds in 4 seconds and invents a deadline; profile B responds in 8 seconds and preserves all the data. Which one should you choose for a table that will be used as a commitment to customers?
Download practice sheetCheck the commented answer
Choose B in this test because fidelity is a required criterion and A failed it. A’s shorter time doesn’t compensate for the invented commitment. Record that the timings are simulated and that the conclusion applies to this set of inputs, not to all tasks.
Check your understanding
A short response worked. Does that prove complete compatibility with an agent?
Key concepts
What stays from this module
A comparison matrix and an example profile without credentials.
- ✓ It doesn’t make up missing data.
- ✓ It executes the required functions.
- ✓ The previous profile can be restored.
Progress records your reading. The practice is completed when you check the submission.