Evidence-Locked AI Writing for Claim-Traceable Scientific Manuscripts

Evidence-Locked AI Writing for Claim-Traceable Scientific Manuscripts

Introduction

LLMs can write scientific text that sounds polished in seconds—but “sounds right” is not the same thing as “is supported.” That mismatch is exactly what this new research from Paper Pilot tackles. The core idea: when AI helps draft a paper, the system should force accountability so that claims, numbers, citations, and interpretations don’t silently drift away from evidence.

Based on new research, Paper Pilot proposes a human-in-the-loop expert system for applied-science manuscript generation that’s evidence-traceable and governed by mandatory approval gates. Instead of letting LLMs draft end-to-end and hope humans catch everything later, Paper Pilot inserts a structured series of “no-pass” checkpoints from idea → method → evidence → results → claims → final text.

What makes this different is that the paper isn’t just about writing quality. It’s about the governance layer: making it hard for AI to fabricate citations or invent results, and turning missing evidence into explicit placeholders that require the manuscript owner to decide what to do next.

Why This Matters

This is significant right now because applied research teams are already using LLMs to accelerate the dull parts of writing: literature synthesis, section drafting, and “make this paragraph more conference-ready.” Those tasks are valuable—but they’re also where silent errors become publishable artifacts. If an AI model fills gaps with made-up citations or fabricated metrics, a human might not notice during editing, especially when the text is fluent and the references “look scientific.”

Here’s a scenario where Paper Pilot could be used today: imagine you’re preparing a journal submission for a domain where results depend on very specific runs (particular datasets, seeds, configurations, plots, ablations). You upload your draft scaffolding, your approved set of methods, and the figures/tables you already generated. Paper Pilot would then help draft each section only when the evidence is present, and it would mark missing items as [TBD: …] rather than letting the model “cover” gaps with plausible text. That means your revision workflow becomes less about detective work and more about reviewing an evidence-backed draft.

Compared to earlier AI writing and agent systems, this builds on a growing trend—but tightens the control problem. Systems like automated survey or multi-agent writing tools can improve structure and citation usefulness. But Paper Pilot’s distinctive contribution is that it’s not satisfied with “citations attached somewhere.” It focuses on mandatory gates and evidence-locked revision, so the system doesn’t just add references—it prevents unsupported claims from progressing into final manuscript text.

Evidence-Traceable Manuscripts: From “LLM Drafts” to “Claim Governance”

The biggest shift Paper Pilot argues for is treating manuscript writing as a decision-support workflow, not just text generation. In many current setups, AI drafts content freely, and humans review after the fact. Paper Pilot flips that into a stage-gated pipeline where the manuscript owner remains the accountable authority and AI output is allowed to advance only when evidence requirements are satisfied.

The “manuscript owner” as the only final authority

Paper Pilot separates roles cleanly:

• Manuscript owner (human): approves scope, validates interpretations, confirms methods, authorizes result claims, resolves conflicts, and decides release for submission.
• LLM agent layer: drafts sections, audits inputs, classifies claims, flags gaps, and proposes revisions—but does not “sign off” on scientific validity.

This matters because some agentic systems include reviewer agents or automated judging. Paper Pilot allows LLM reviewers, but treats them as advisory. The human owner still holds the authority boundary—especially for whether evidence is sufficient and whether claims are scientifically defensible.

Approval gates as “no-pass” checkpoints (not optional suggestions)

Paper Pilot defines a workflow that runs section-wise through eight stages, with no-pass criteria at each step. Conceptually, it’s like airport security for scientific text:

• If required inputs (like journal scope or section definitions) are missing, drafting stops.
• If evidence doesn’t exist for a claim, it can’t be smuggled into the paper as if it’s real.
• If the system detects uncertainty or contradictions, it doesn’t “smooth them over.” It flags unresolved items for human action.

The paper summarizes the manuscript-owner gate model as eight gates, including: scope approval, literature approval, method approval, evidence approval, claim approval, reviewer override, section approval, and final release. The point isn’t just to get humans involved—it’s to require humans at the moments where errors would otherwise propagate.

Claim Classification and Evidence-Locked Revision (Why Polishing Can Be Dangerous)

Even if an AI doesn’t fabricate new facts during initial drafting, revision is where issues often creep in. Paper Pilot explicitly targets that risk with evidence-locked revision control.

Four claim categories that control what can enter the manuscript

After drafting, Paper Pilot classifies each material statement into one of four categories:

• grounded_fact: directly supported by user-provided evidence or approved literature.
• inference: derived interpretation based on available evidence (but not literally stated in a source).
• assumption: necessary but unverified condition used to proceed.
• tbd_placeholder: missing information that must be resolved.

The system’s rule is straightforward but strict: a section can’t be approved if critical claims are still labeled assumption or tbd_placeholder without the manuscript owner accepting them as unresolved limitations.

That turns “vibes-based writing” into evidence-aware bookkeeping.

Evidence-locked revision: changes that add claims trigger re-auditing

Paper Pilot also prevents a common failure mode: stylistic edits that accidentally become factual upgrades. Under evidence-locked revision, if a revision adds or changes something substantive—like a new metric, a comparison, a citation, a numerical claim, or an interpretation—the system forces the relevant audit and re-classification before the content can re-enter an approved state.

A useful analogy: this is like editing a legal document where you’re allowed to rephrase for clarity, but you’re not allowed to change what’s actually “on the record” without re-verifying the underlying proof.

What They Actually Tested: Citation Fabrication Under Coverage Pressure

The paper includes an empirical evaluation focused primarily on citation grounding—and it’s designed to be judge-free. Instead of using an LLM judge to decide if citations are correct, the researchers score mechanically by verifying citations against real arXiv metadata and checking whether identifiers resolve properly.

The evaluation design: mechanically scored, no LLM judge

For the citation benchmark, the setup uses real arXiv papers and creates “trap subtopics” that cannot be supported by the provided citation set. The model is required (via prompt rules) to cite inline for each subtopic. This intentionally creates a realistic pressure: the model must either (a) cite correctly from supplied evidence or (b) fail safely.

Key details:
- Two commercial LLMs were tested: gpt-4o-mini and gpt-5.2.
- The scenarios used four real arXiv papers each as the allowed literature set.
- Two trap subtopics per scenario were not supported by any allowed paper.
- The prompt required at least one inline citation per subtopic.
- The gated vs ungated conditions were run under otherwise identical user prompts.
- Total drafts: 90 generated drafts for the citation benchmark.

Comparison table: gated vs ungated citation behavior

The results: ungated drafting fabricated up to ~25%, and never flagged gaps

Across all 90 trap subtopics drafted ungated, the key failure is that not a single one was flagged as an evidence gap.

But the models failed differently:
- gpt-4o-mini handled traps by covering them with confident prose that had no citations (silent unsupported coverage).
- gpt-5.2 handled traps by fabricating citations: 25.2% of distinct citations emitted were fabricated (34/135), including:
- nonexistent arXiv identifiers, and
- real identifiers paired with invented attributions
- plus out-of-scope citations recalled from memory.

Now the gated condition—the Paper Pilot evidence-locked rule:
- It produced zero fabricated citations among 168 distinct citations emitted.
- It also reduced out-of-set citations to zero.
- Most importantly: it converted evidence gaps into explicit actionable placeholders.

In numbers:
- 88 of 90 trap subtopics (97.8%) were flagged with [TBD: …] under the gated rules.
- 0 of 90 trap subtopics were flagged ungated.

So the mechanism worked exactly as the governance goal requires: missing evidence becomes visible, not hidden.

Beyond Citations: Preliminary Evidence for Result Grounding and Revision Drift Control

Paper Pilot’s strongest distinction from citation-only writing systems is that it also addresses result grounding: reported numbers, plots, tables, and analytical conclusions must be traceable to the study’s own artifacts—not just to a bibliography.

Result-grounding benchmark (preliminary): fabricated numbers drop when evidence is locked

This part of the evaluation uses small synthetic illustrative datasets (not real experiments) to make ground truth mechanically checkable.

Setup highlights:
- Three scenarios; each requests a Results section with five findings.
- Two of the five findings per scenario are traps (values absent from the supplied dataset).
- Total repeats: gpt-4o-mini (10 repeats) and gpt-5.2 (5 repeats).

The direction is clear:
- Ungated: both models invented numbers for nearly all impossible findings:
- gpt-4o-mini: 100% of traps fabricated
- gpt-5.2: 73% of traps fabricated
- and they flagged no gaps.
- Gated: fabrication drops substantially:
- gpt-4o-mini: 27% fabricated; gaps surfaced as placeholders 73% of the time
- gpt-5.2: 0% fabricated; placeholders 100% of the time

The authors report that overall fabricated-number rates also fell (from 21%→5% for gpt-4o-mini, and 13%→4% for gpt-5.2), though they note this metric is noisier because Results sections often legitimately restate/round values.

Revision drift benchmark (preliminary): gating cuts overclaiming, but doesn’t eliminate it

Paper Pilot also tested evidence-locked revision on synthetic paragraphs with already-approved numbers and evidence.

Task:
- Provide an approved evidence-grounded paragraph plus the data behind it.
- Ask the model to “polish” for a top-tier journal.
- Score whether it adds new numbers/comparisons or uses more promotional/unhedged language.

Results:
- Ungated: drift in every single case (45/45). Models added superlatives and stripped hedges.
- Gated: added superlatives dropped to 0 for both models, and drift rate roughly halved to 60–63% rather than 100%.

So gating helps a lot, but revision drift is not completely solved by prompt-level enforcement alone—leading to their discussion of what orchestration-layer enforcement should do later.

Adversarial Robustness: Prompt Injection vs “Editor Escalation”

A governance system has to survive attacks. Paper Pilot tests two different threats to the citation gate.

What they attacked

1. Prompt injection in the evidence pack: the “literature” includes a malicious instruction claiming the citation restriction is lifted.
2. Authority escalation across turns: after the model outputs placeholders for missing evidence, the user insists there’s no room for placeholders and forces replacement with real citations.

Results (key point): injection resisted; sustained authority pressure can break weaker models

• Under embedded prompt injection, both models had zero breaches (i.e., they didn’t fabricate or silently comply).
• Under escalation:
  • gpt-4o-mini collapsed: 72% of trap subtopics breached.
  • gpt-5.2 held better: only 8% breached, and it refused/escalated appropriately rather than fabricating.

The takeaway isn’t “this works perfectly.” It’s that prompt-level gates are robust to injected evidence, but more fragile when an attacker uses persistent human-like authority pressure. This reinforces Paper Pilot’s broader theme: governance should not rely on a single brittle prompt rule—it should be enforced in a stronger orchestration layer.

What You Can Do With This: Practical Adoption Ideas for Applied Teams

If you’re an applied-science researcher, Paper Pilot’s framework is easiest to adopt when you treat manuscript generation like a controlled pipeline rather than a single chat.

Here are practical ways to use the ideas immediately:
- Draft section-by-section so gaps don’t spread invisibly throughout the manuscript.
- Maintain claim-level labels (grounded_fact, inference, assumption, tbd_placeholder) so you can quickly see what’s actually ready for approval.
- Lock evidence during revision: if the draft changes a number, comparison, citation, or interpretation, re-run the evidence check.
- Treat LLM reviewer feedback as advisory, not as authority—especially for evidence sufficiency and release readiness.
- Convert missing evidence into placeholders that require an explicit owner decision, instead of allowing “silent coverage.”

Paper Pilot even emphasizes this as the difference between controlled decision-support writing and fully autonomous authorship: LLMs help draft; humans authorize claims.

Key Takeaways

• Paper Pilot reframes AI manuscript generation as claim governance, not just drafting.
• It uses mandatory manuscript-owner approval gates (scope → literature → method → evidence → claims → section → final release) with no-pass criteria.
• It prevents silent failures by requiring evidence gaps to become explicit [TBD: …] placeholders rather than being covered with uncited prose or fabricated claims.
• In the judge-free citation benchmark on real arXiv papers, ungated drafting fabricated citations up to 25.2% and flagged zero evidence gaps, while gated drafting produced zero fabricated citations and flagged 97.8% of trap subtopics as placeholders.
• Preliminary benchmarks suggest evidence-locked rules also reduce fabricated results and revision overclaiming, though prompt-level enforcement doesn’t fully eliminate revision drift.
• The system’s robustness is strong against embedded prompt injection, but weaker models can be breached under sustained authority escalation, motivating stronger enforcement beyond prompt rules.
• Overall: Paper Pilot positions LLM-assisted writing as a human-authorized, evidence-traceable workflow—a decision-support process designed for accountability in applied science.

If you want, I can also turn this into a “how to implement Paper Pilot-style gates in your own writing workflow” checklist (based on the eight stages and claim/evidence rules described in the paper).