How to Test an AI Agent Properly

Step 1: define the job before choosing the model

Write down the real task classes the agent must perform: reasoning, code generation, file operations, structured data, browser actions, device control or long-running workflows. A benchmark should represent the job, not merely produce a convenient score.

Step 2: freeze a small reproducible suite

Use fixed prompts, expected outputs and scoring. Version the suite. If tasks change between runs, you are comparing stories rather than systems.

Step 3: score model capability separately

Measure whether the model can reason about the task or produce the required structure. Keep this separate from transport and tool failures.

Step 4: record the provider route

Store model and provider together. Record HTTP status, latency, empty responses, timeouts and rate limits. A provider failure should not silently become a model-quality failure — or disappear from the report.

Step 5: verify structured output

Use parsers, JSON Schema, type checks or exact contracts. Reject malformed tool arguments before execution.

Step 6: verify permission boundaries

Test both allowed and denied actions. A permission system is not proven if you only test requests that should succeed. Include attempts to write outside the workspace, run ungranted processes or access capabilities without consent.

Step 7: execute real tools against safe targets

Use isolated test repositories, synthetic files, local fixtures or a dedicated device. Confirm the external state after execution. A model saying “done” is not evidence.

Step 8: force failures

• return a tool error,
• disconnect the provider,
• send a 429 or timeout,
• remove a file between plan and execution,
• deny permission,
• return an HTTP 200 with invalid payload,
• interrupt a multi-step workflow.

Observe whether the agent stops, retries, changes route, asks for approval or corrupts state.

Step 9: test state transitions

Represent at least success, failed and unknown outcomes. For multi-step work, test continuation after interruption and ensure completed actions are not repeated blindly.

Step 10: repeat enough times to expose route instability

One pass proves a path exists. Repeated fixed runs reveal variance, provider instability and nondeterministic tool behaviour. The repetition count should match the risk and workload rather than an arbitrary universal number.

Step 11: use stronger verifiers as tasks get closer to production

For code, compile and run tests. For files, hash or read back. For APIs, validate the response and resulting server state. For UI, inspect the resulting state rather than assuming a click worked.

Step 12: publish the limitations

State the device, app/build version, provider route, model identifier, task count, verifier and important missing gates. This prevents a narrow PASS from becoming an accidental universal claim.

A useful evidence record

The SC LABS shorthand

For real agent tests we think in this chain:

Prompt → Model → Provider → Permission → Tool → State → Verification → Recovery → Repeat.

If the report skips one of those layers, be careful about what the final score is allowed to claim.