Testing AI agents and MCP servers: a QA framework for tool-using AI

Why agent testing is a different problem

Our approach to testing AI features covers single-turn generation, hallucination, and guardrail checks against golden datasets. An agent adds a second dimension entirely: it plans a sequence of actions, calls tools through MCP servers, observes results, and decides what to do next, often across many steps. A wrong decision at step three does not look wrong in isolation; it only becomes visible when the final outcome is checked, by which point the agent may have already written data, sent a message, or spent real money.

The four layers to test

We split agent and MCP testing into four layers, tested largely independently before being tested together end to end. Skipping straight to end-to-end testing is the most common reason teams struggle to isolate failures once something goes wrong in production.

Layer 1: the MCP server itself

Before any agent touches it, an MCP server is just an API and should be tested like one: does each exposed tool return the correct result for valid input, does it fail gracefully and informatively on invalid input, and does it enforce the permission scope it claims to enforce. This is standard API and contract testing, and skipping it because "the agent will figure it out" is how a malformed response quietly becomes a hallucinated action downstream.

Layer 2: tool-call correctness

Given a specific prompt and context, does the agent choose the right tool, with the right arguments, and does it recognize when no tool call is the correct answer? We build a fixed set of scenario prompts with a known-correct tool call (or deliberately no call) and score the agent's actual choice against it on every model or prompt change, the same golden-dataset discipline used for any AI feature evaluation, applied to actions instead of text.

Layer 3: multi-step state

This is where agent testing diverges most from single-turn testing. Does information from step one correctly reach step four? Does the agent recover sensibly if a tool call in the middle of a sequence fails or times out, or does it silently proceed on stale or missing data? We test this with deliberately injected mid-sequence failures, a tool that times out, a resource that returns empty, because production will do this eventually and the recovery behaviour has to be verified, not assumed.

An agent that handles the happy path perfectly and falls apart the first time a tool call times out has not been tested, it has been demoed.

Layer 4: safety and permission boundaries

Guided by the same spirit as the OWASP LLM Top 10, we adversarially test whether prompt injection through a tool's own returned data (a poisoned document, a malicious ticket description) can make an agent call a tool it should not, and whether the MCP server's own permission scoping actually holds when an agent tries something outside its intended lane. Security here has to be verified at the server boundary, not assumed from the model's good judgment.

Building an agent eval suite

Bring the four layers together into one suite that runs on every change: server contract tests, tool-selection scenarios, multi-step state and failure-injection scenarios, and adversarial permission tests, all scored automatically so a regression is caught before a release, not after a customer reports strange behaviour. If you are running multiple agents, this suite needs to run at the individual-agent level and again at the full system level, since a passing agent in isolation can still fail once it is coordinating with others.

Shipping an agent or an MCP server and want a QA framework built around it before launch? A free 30-minute testing audit will tell you exactly where the risk is.