In August I published fault-mcp, an MCP server that breaks its own tools on purpose. One of its 6 faults is schema: the tool answers with a payload that does not match its declared shape. For example a number arrives as a string or a required field is missing. The README has a design note about it:
A client that validates structured output rejects a
schemafault at the transport boundary, so the agent never sees it and you end up measuring the client instead of the agent.
So the server hides outputSchema by default and there is a flag, --advertise-output-schema, to publish it. The repository has a test that the schema is hidden or listed. No test sends a schema fault through a client with the schema published. I never ran it through the agent I use every day either. The sentence names a trap without saying which clients fall into it and I took it as true because it sounded right.
The setup
fault-mcp 0.1.0 from npm, the tool get_account_balance, account ACC-4471. The clean answer is 4,071,877 minor units, so $40,718.77, in structuredContent and as one line of text. I wrote 9 profiles with probability 1: a clean one, error, empty, truncate, 2 corruption strategies and 3 schema modes. I left latency out, it gives the right answer late. Each profile ran with the schema hidden and published.
On the SDK level there were 3 client libraries in 4 setups. Raw JSON-RPC over stdio without any SDK shows what the server put on the wire. The TypeScript Client from @modelcontextprotocol/sdk 1.32.1, once calling callTool directly and once after listTools(). And ClientSession from the Python mcp 2.3.0. Then the agent: Claude Code 2.1.292 with claude-opus-5, no built in tools, only this one MCP server, started from a fresh folder in /tmp with only the MCP config in it. Its stream log records the tool result as Claude Code passed it to the model. The server and the SDK clients ran on Node 22.23.1.
What the SDK clients did
With the schema hidden all 5 clients, Claude Code included, returned whatever the server sent. The only exception is wrong-container, it has its own section below. With the schema published the picture splits.
The TypeScript client after listTools and the Python client threw on 4 of the 8 faults. 2 of them are the schema faults the README talks about. The other 2 surprised me: empty and truncate. Both drop structuredContent and a tool with an output schema must return it. Both clients raise “has an output schema but did not return structured content”. So truncate, which is meant to be a quiet fault, becomes an exception before anything reads it.
The 2 corruptions passed every client in both modes. The plausible one returns the row of a neighbouring account under the requested id: $9,381.67 with the right types and fields, but for another account. A validator checks only the shape and the shape was fine.
The TypeScript client without listTools did not check anything against the output schema. I looked into the SDK code. The validators are built in cacheToolMetadata and that runs only when listTools() returns. callTool on its own finds no validator and returns the result. Python lists the tools itself when the name is unknown, so it validates either way. An eval harness that calls a tool by name without listing first gets the lenient behaviour from the strict SDK.
The row nobody received
wrong-container put an array where the object was. It never reached any client, with or without the schema. Raw JSON-RPC got error -32602 “Invalid tools/call result” from the server itself.
faultmcp is built on the SDK’s low level Server. The README even explains why: a fault injector must be able to return payloads a strict result type would refuse. But that Server wraps every tools/call handler and parses the result with CallToolResultSchema before sending it. The cast in TypeScript only silences the compiler. The runtime check in Server still runs. I checked 1.30.0, the lowest version the package.json of faultmcp allows. It does the same.
So one of my 3 schema modes has not worked through the transport on any version I checked. Its unit test calls applyFault directly and passes. I had 85 green tests and none of them sent this fault over a pipe.
What the agent did
Claude Code was the client I cared about most, because it is the agent I use. I put a small tee between Claude Code and the server to see the wire. tools/list carried the full outputSchema, with balance_minor declared as an integer and all 4 fields required.
Then the type-swap fault sent "balance_minor": "4071877" as a string. Claude Code passed it to the model without any complaint. drop-required removed account_id and that passed too. Same with the schema hidden. Claude Code did not validate the structured output in any of my runs.
The model answered $40,718.77 in all 6 runs with these 2 faults. It never mentioned the missing account id or the number in quotes. In every run that gave a number it did warn that the as_of date was months old, but that is just the date in my fake ledger and not a fault. With empty, truncate, error and wrong-container it said every time that there was no balance. The plausible corruption went through as $9,381.67 both times.
I did not expect one more difference between clients. When structuredContent exists, Claude Code gives the model its JSON and not the text block. The generic corruption rewrites the text and the structured payload separately, so the text said ACC-4476 while the structured part said ACC-9471. The model saw ACC-9471 and caught the wrong account both times. A harness that reads content[0].text would get the true $40,718.77 under ACC-4476, while the model in Claude Code got ACC-9471 with $43,035.36.
So what does the flag measure
The README is right that a strict client can eat the fault before the agent sees it. But the protocol leaves it open. The spec version Claude Code negotiated, 2025-11-25, says clients SHOULD validate structured results, not MUST. The official TypeScript SDK is strict only after listTools, Python is always strict, Claude Code is not strict at all. With my server, a published schema changed 4 of 9 results in 2 clients and 0 of 9 in the agent.
The default to hide the schema stays, because with it hidden every client I tried got the same result. But the reason in the README should name which clients validate and say that empty and truncate turn into exceptions too. wrong-container has to be fixed or removed, because right now it tests the server SDK. I have not pushed any of it yet.
What I did not check
The agent ran once per cell plus 2 extra runs with the tee. So the model numbers are counts of a handful of runs and not rates. I used only 1 model, claude-opus-5. I did not try Cursor, Codex CLI or any other agent and they may validate. Both SDKs were the latest releases on the day. The 2.x line of the TypeScript client, @modelcontextprotocol/client 2.3.1, I checked only on 3 faults. It behaved like 1.32.1, strict only after listTools. I only used stdio, not HTTP.