Agents fail at handoffs: seven checks from new research

In one production agent's table study, delivering rows over a typed side channel instead of the model's context lifted fully correct turns from 45.7% to 92.3%. In the handoff and table studies, the fixes that worked put structure or a deterministic check around the model.

The papers cover citation fabrication after a citation request, poisoned tool descriptions in MCP, signed requirement handoffs between agents, data corruption when an agent re-reads big tables, policy gates for coding agents, duplicated agent instruction files and multi-tool retrieval.

Each is an arXiv paper; we have not checked whether it has been peer reviewed. Only the abstracts were available to us, so we report what they state and nothing about methods beyond that.

Asking for citations made post-trained open models fabricate far more

The study ran 900 frozen questions across US federal case law, computationally adjacent scientific literature and medical literature. It used twelve open-weights checkpoints covering four post-training stages of three model families, then refit on fifteen. Holding the weights fixed and changing only the prompt, asking for citations raised fabrication sharply in post-trained checkpoints and not in base ones. The interaction (how much more the citation request raised fabrication in post-trained than in base checkpoints) was 47.8 points on twelve checkpoints and 41.5 on fifteen, in all three domains.

A length-matched control separated the citation cue from a demand for a longer answer, so the effect is tied to citations, not verbosity. Post-training also raised fabrication overall, from 4.8% to 56.7%. The author's own verdict on whether this is reward hacking is "possibly, but not proven": the registered prediction that fabrication would rise most where it is most tempting did not hold, and plain instruction-following could explain the same results.

**What to do:** run 20 of your own questions twice, with and without "cite your sources," and look up every citation in the second run by hand. The checkpoints were open-weights models, so this does not tell you how ChatGPT or Claude behave on your questions.

Tool descriptions are an attack surface, and the model matters more than the framework

The Model Context Protocol (MCP) lets agent frameworks connect models to external tool providers, and it carries attacker-controlled content into the model's context without validation. The authors built an open-source tool, mcpvenom, and filled a 9,720-cell matrix: 12 frameworks, 18 models and 45 attack scenarios. Exfiltration counted only when a canary HTTP request confirmed it, so this measures real execution, not just whether a model agreed to follow a bad instruction.

The overall attack success rate was 22.9%, or 2,113 vulnerable cells. Of those cells, 64.9% dispatched an attacker tool call, 23.0% contacted the attacker's endpoint and 12.1% verifiably exfiltrated a secret. Nine core frameworks clustered within 23 to 26%, and a code-level audit found no intentional defense in any of the twelve. Model attack success ranged from 0.4% to 49.7%: Claude Opus 4.8 was vulnerable in 8 of 530 cells while issuing 223 explicit refusals. A regex description scanner rejected 20 of 45 scenarios at a 7.2% false-positive rate, so scanning alone is a partial control.

**What to do:** do not assume your framework protects you; it probably does not. Pick the model with attack resistance in mind, treat every tool description from a third-party server as untrusted input, and add scanning only as one layer.

Signed requirement manifests stop tampering in transit, not at extraction

When an orchestrator splits a request across subagents and merges the results, any handoff can summarize away, alter or ignore a requirement. The study used synthetic, exactly scorable tasks and compared free-text, structured JSON and signed-manifest handoffs. In the signed version, the orchestrator extracts each requirement as a verbatim quote plus a checkable predicate, code signs the manifest with HMAC-SHA256, and deterministic code verifies the merger's decision.

With Qwen3-8B (1,440 trajectories, five threats), signed manifests cut unsafe commits under attack from 69.4% to 9.0%, with no net loss of conformant commits relative to structured handoffs. An unsigned manifest committed violations in 79% of mutated runs; the signed one in none. On identical trajectories, the deterministic verifier caught 78.7% of violating decisions against 59.0% for an LLM checker, with 7% versus 50% false alarms.

The boundary is extraction. One adversarial line injected before signing raised signed-manifest violations from 14.6% to 37.5% (Qwen3-8B), 11.1% to 63.9% (gpt-oss-20b) and 8.3% to 75.0% (GPT-5.6-luna). The signature then authenticated the corrupted manifest.

**What to do:** write requirements down as a quote plus a check that code can run, and have code verify the final decision instead of asking another model. Then review the extraction step by hand, because that is where the remaining risk sits.

Big tables pasted into context collapse; let a filter answer the follow-ups

The authors measured a production award-travel agent in two architectures over identical frozen payloads, with lineage-matched prompts and sampling. SYNTHESIS feeds full record sets into the planner's context and answers follow-ups in-head. BYPASS, the deployed system, keeps records in session state, turns follow-ups into arguments for a deterministic filter, and delivers rows over a typed side channel.

Across 24 dialogues (339 scored turns per condition), BYPASS scored 92.3% of turns fully correct and SYNTHESIS 45.7%, a paired gap of +46.6 points (interval +35.7 to +57.1). Table size drove the drop: SYNTHESIS was near-perfect at 10 rows or fewer and fell to 8% or less beyond about 74 rows. The collapse threshold ordered by model capability, from under 10 rows (GPT-4o-mini) to about 198 or more (Gemini 2.5 Pro), while BYPASS stayed flat in size for all four models. Beyond 100 rows, each correct SYNTHESIS answer cost roughly 200 times BYPASS's flat $0.012.

**What to do:** if your agent re-emits tool output across turns, count the rows. Past about 74 rows, move filtering, sorting and lookups into code and let the model write the arguments.

Examples and counterexamples make policy gates catch more and interrupt less

Coding agents can delete data, leak secrets or message the wrong person faster than anyone can review. The study evaluated 22 hosted decision models and two prompted general models as classifiers for a 28-rule policy engine. The dataset was 2,643 synthetic tool calls labeled blind against the rules, with a sealed holdout.

With one shared preamble in both conditions, writing each boundary as examples and counterexamples raised recall by 5.5 to 26.1 percentage points on all 23 available models and reduced false positives on 21. Jev 1.14 went from 89.3% to 98.2% recall, while unnecessary interruptions fell from 30.6% to 4.8% of 252 clean actions. With examples, its median latency was 120 ms at $0.46 per thousand requests. Strong models kept the recall gains when concrete names were replaced by generic descriptions, and three largely kept them when the boundaries were written as prose. On 16 new scenarios written after the initial studies, a follow-up improved recall on 22 of 23 models.

**What to do:** for every rule in your approval gate or system prompt, add one call it should block and one near-miss it should allow.

Some of your instruction files may disagree with each other

The study examined 3,387 instruction files from 1,000 GitHub repositories across four formats: AGENTS.md, CLAUDE.md, GitHub Copilot instructions and Cursor rules. In the higher-popularity cohort, 42.2% of instruction-bearing repositories maintained more than one format. Among 386 manually examined pairs of corresponding sections, 16.1% gave divergent guidance. Also, 40.4% of the analyzed files had only one recorded modification event.

From category prevalence the authors derived a template of ten Required, four Gap and two Recommended categories. In an exploratory generation experiment, explicit template guidance raised structural category coverage from 18.7% to 82.8%. Coverage is not fidelity: in a manual assessment of 278 generated category instances, 19.4% fully preserved developer-written intent and 61.2% partially preserved it.

**What to do:** if you keep more than one of these files, diff the matching sections today and pick one as the source of truth. Treat generated instruction files as drafts to read, not as finished.

A tool retriever has to find every tool, not most of them

An agent that needs several tools fails if the retriever omits any one. The authors compared six retrieval structures on two multi-tool testbeds: 345 AppWorld queries and 103 MCP-Bench tasks over 257 MCP tools. Their metric, completeness@k, is the share of queries whose entire required tool set appears in the top k results. On all 7,961 ToolRet queries, completeness at 10 was .58 to .67 for single-target queries and .08 to .17 for queries with four or more targets.

No untrained structure making at most one LLM call per query completed more than 10% of AppWorld queries at k = 16. A set-level retriever trained on 75 to 90 in-domain queries reached .33 on AppWorld and .58 on MCP-Bench with no LLM calls. An LLM-navigated hierarchy reached .30 and .40 but needed a median of 31 to 38 LLM calls and about five minutes per query. In a downstream check with the official AppWorld agent, tasks with a complete retrieved set succeeded 21 of 21 times; incomplete ones succeeded 2 of 39 times.

**What to do:** score your retriever on whole tool sets for your multi-tool requests, and record calls and latency next to the score.

The seven papers side by side

What changes in how you prompt and build agents

Stop asking the model to guard its own boundaries. Two of these papers (handoffs and table delivery) point the same way: let code hold the data, check the decision and filter the rows, and use the model to translate intent into arguments.

Two prompting habits are worth changing now. Do not add "cite your sources" by reflex; test it against your own questions first. And when you write rules for a system prompt or a gate, include examples and counterexamples, since explicit boundaries written that way raised recall in the decision-model study.

For agent builders, the checklist is short. Review MCP servers as untrusted input, run completeness checks on tool retrieval, and keep one instruction file as the source of truth.

Which result we would trust most

We would trust the table-delivery result most. Both architectures saw identical frozen payloads with lineage-matched prompts and sampling, so only delivery differed, and the interval on the gap excludes zero. The mechanism is also easy to check yourself: feed the same table at different sizes and watch where answers break.

We would trust the citation study least as a basis for action, not because it is careless but because its author concludes "not proven." The MCP measurement ranks next, since real execution confirmed by canary receipt is a stronger standard than counting compliant replies.

Limits to keep in mind

Most results come from narrow settings. The handoff and policy-gate studies used synthetic tasks, the table result comes from one award-travel agent, and the citation study used open-weights checkpoints. The signing result was measured on Qwen3-8B with 1,440 trajectories, so treat it as a design pattern to test, not a guarantee. The instruction-file study describes GitHub repositories, and the reported divergence is a count of section pairs, not evidence that agents behaved worse. We have read only abstracts, so details such as how fabrication was scored are not covered here.

You are reviewing the design of an AI agent workflow. Work in two steps. First, write a short plan listing which of the five areas below apply to my description. Then review only those areas.

My workflow: [DESCRIBE THE AGENT, ITS TOOLS, AND WHAT DATA IT HANDLES]
Third-party tools or servers it connects to: [LIST, OR "NONE"]
Largest table or record set it may handle: [APPROXIMATE ROW COUNT]

Areas:
1. Untrusted input: which tool descriptions or tool outputs could carry attacker text?
2. Handoffs: where could a requirement be summarized away or altered between steps?
3. Data fidelity: where would the model re-read or re-emit large records instead of code filtering them?
4. Risky actions: which actions need a gate?
5. Tool coverage: which requests need several tools at once?

For each area, give: the risk, one check that deterministic code can run instead of the model, and one example of a call to block plus one near-miss to allow: [EXAMPLE OF A RISKY ACTION] / [EXAMPLE OF A SAFE ACTION].

Rules: quote the exact phrase of my description each point relies on. Do not cite outside sources, papers or standards. Say "unknown" instead of guessing. End with 10 outputs I should check by hand.