Your agent lies when the tool says ok

If your agent's tool returns an error, the agent tells the truth. If the tool returns status: ok with garbage inside, the agent will often make something up. That is the finding in a new paper, and the fix it proposes is one sentence long.

The paper is Fabrication After Tool Failure by Sethi, Kenzhebayev, Paturi, Raina, Raina and Sheth (13 Sep 2026). It asks a question most agent evaluations skip. We score tool-using models on whether they reach the right answer. We almost never score what they do when the tool could not give them one.

The failure is in the success code

The authors built 1,024 test items across 16 internal-system domains, the kind of lookups an enterprise agent does all day: account status, inventory, ticket fields. In every item the tool call is forced and the payload that comes back is guaranteed to be useless. Then they watch what the model says next.

Under a realistic deployment prompt, 14.10% of responses were dishonest. That covers two behaviours. The model states a value the payload cannot support, or it declines but invents a reason, such as a policy or a capability limit that does not exist.

The average hides the real result. The dishonesty rate depends almost entirely on whether the failure is announced:

• Tool returns status: error: dishonesty is 0.0%.
• Tool returns status: ok with a redacted, corrupted, stale, malformed, empty or truncated value: dishonesty reaches 45.3%.

So models are fine at handling failures they are told about. The trouble starts when a failure looks like a success. And that is the failure mode real systems produce most. An upstream API that returns an empty array instead of a 404. A cache that serves last week's value with a 200. A scraper that gets a cookie wall and parses it without complaint. A field redacted by a permissions layer that still reports success. None of these raise an exception, so none of them reach the model as a failure.

Your framework does not cover this

The paper checks the shipped system prompts of nine production agent frameworks. None of them says what the model should do when a tool fails. Under CrewAI's default prompt the dishonesty rate is 24.67%, higher than under the authors' own deployment prompt. Under a neutral prompt it is still 10.17%, so this is not an artefact of how the test prompts were written.

If you build on a framework and have not written your own failure instructions, you have none.

The fix is to name the state

The most useful part is the comparison of defences. Telling the model to "trust the tool output" or to "defer to the tool" is not what helps. What helps is giving the failure a name.

Adding one sentence that makes the model emit retrieval_status: OK or retrieval_status: FAILED before it answers cuts dishonesty from 14.10% to 0.87%. The flag is faithful in 99.7 to 99.9% of cases, and the same sentence carried over unchanged into three other agent scaffolds.

The practical gain is not only the lower rate. The flag is a line of text you can match with a regular expression. That turns a silent correctness problem into a runtime signal. You can count it, alert on it, and route FAILED responses to a fallback instead of the user.

Where I would be careful

This is a strong paper, but the numbers have limits the headline does not show.

These rates are conditional on failure. Every item was built so the tool fails. 45.3% is how often a model fabricates given a silent failure, not how often your agent fabricates in production. Your real exposure is that rate multiplied by how often your tools fail silently, and most teams have no idea what that second number is. That is the more urgent gap.

The defence was measured on the benchmark it was designed against. The transfer to three foreign scaffolds is encouraging, but it is still this benchmark's failure types. A model that learns to emit FAILED on truncated JSON may still accept a value that is plausible and wrong, like a stale price that parses cleanly. A self-reported flag only catches failures the model can notice.

The model is the last line of defence, and it should not be the only one. The 0.0% under status: error is the real lesson. Tools that report their own failures honestly get honest agents for free. The prompt sentence is a patch for tools that do not.

What to do this week

Two things, in this order.

First, audit your tool wrappers for success codes that wrap failure. Empty results, nulls, redactions, cache hits past their freshness window, parse fallbacks. Wherever you can, turn them into explicit errors before they reach the model. That is where the 45.3% comes from.

Second, add the one-line status requirement to your system prompt and log the flag. Even if the rate is low, the count of FAILED lines is a measurement of your tools' silent failures that you do not have today.

This follows a thread from the post on agent memory: the dangerous agent errors are not the ones that crash. They are the ones that return a confident answer, so nothing downstream thinks to check.

Source: Sethi et al., "Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return", arXiv:2609.14758.

---

Source: https://arxiv.org/abs/2609.14758