Hallucination vs Accuracy: A Practical LLM Trust Check (2025 Bench)

Hallucination vs Accuracy: A Practical LLM Trust Check (2025 Bench)

Introduction

If you’ve ever asked an LLM a “simple” academic question and gotten a confident, polished answer… only to later realize parts of it were wrong, you’re not alone. That gap—fluent but fallible—is exactly what new research from the arXiv paper digs into. The study benchmarks hallucination and factual reliability across ten major LLMs, using 70 carefully designed academic prompts in four difficulty tiers.

What makes this work stand out is that it doesn’t treat “quality” as one number. Instead, it measures multiple angles of trustworthiness—how often models are correct, how much they cover, how frequently they produce unsupported claims, and even how far their confidence/response behavior deviates from expected truth. The evaluation was conducted in February 2025 on specific model versions, with each prompt answered once under default settings and scored independently by three reviewers.

In short: this isn’t just “which model sounds best?” It’s “which model is most reliably factual under pressure—especially when prompts get tricky?”

Why This Matters

Right now, academic workflows are quietly changing. People are using LLMs to draft literature reviews, explain concepts, summarize papers, and generate “supporting” background sections. The risk is that hallucinations don’t always look like obvious nonsense—they can look like a plausible academic paragraph with a subtly wrong citation, a mischaracterized claim, or a made-up detail.

This research is significant now because it’s explicitly about the reliability problem in academic writing tasks, not just generic QA. And it tests reliability across prompt complexity (Basic → Explanation → Advanced → Tricky), which mirrors what happens in real use: you start with straightforward questions, then you move to deeper reasoning, then you hit ambiguous or adversarial phrasing (often without realizing you’ve done it).

A real scenario where you could apply this today: imagine a student or junior researcher using an LLM to produce a section of a thesis about a complex topic. If the model is “highly fluent” but has a high Hallucination Index, the output can get used as a skeleton—then errors propagate into summaries, slides, and even citations. This benchmark helps you select tools and set expectations: some systems are excellent at being complete, but may be more prone to unsupported details; others are conservative and less likely to fabricate, but may omit important content.

Finally, how this builds on earlier AI research: prior work often looked at hallucination types or used broad general-purpose benchmarks. This study adds something more practical: a multi-metric, academic-writing-focused comparison across both proprietary and open-source models, explicitly tracking the trade-off between completeness and factual reliability—and showing that prompt difficulty plays a huge role. That combination is exactly what many earlier benchmarks were missing.

What the Researchers Actually Measured (and How)

This paper turns “trust” into measurable components. Each model response is evaluated against benchmark answers at the level of a single model response to a single prompt. That means the study isn’t relying on one long document-level judgment; it evaluates 700 total items:
- 10 LLMs × 70 prompts

The 70 prompts, split by complexity

The benchmark prompts are grouped into four tiers:
- Basic: straightforward factual questions, definitions, simple retrieval
- Explanation: conceptual understanding and structured explanations
- Advanced: multi-step reasoning and deeper analysis
- Tricky: ambiguous/adversarial prompts designed to trigger hallucination (including a structure where a false clause can be embedded alongside a true one)

A key point: this isn’t “hard for the sake of hard.” The Tricky prompts are designed to test whether a model can handle embedded falsehoods rather than accepting the whole statement.

The five evaluation metrics: accuracy, reliability, and hallucination risk

Each response gets scored on five metrics:
- Accuracy: proportion of responses that are factually correct
- Precision: when the model gives an answer, how much of it is actually correct?
- Recall: how much of the correct benchmark content the model covers
- Response Deviation Index (RDI): a calibration-style gap between response score and ground truth
- Hallucination Index (HI): essentially the fabrication rate expressed as a percentage

Importantly, scoring wasn’t done by a single judge. The study used three independent reviewers, and it reports strong agreement:
- Intraclass Correlation Coefficient (ICC(3,1)):
- Accuracy 0.81, Precision 0.79, Recall 0.83, RDI 0.76, Hallucination Index 0.84
- Krippendorff’s alpha (ordinal reliability): 0.79

So the benchmark is meant to be more reproducible than “one evaluator vibes.”

Accuracy, precision, recall, and hallucination—why the trade-off matters

Think of this like a medical test:
- Precision: if the model claims something is true, how often is it actually correct?
- Recall: does the model capture all the right parts, or does it skip items?
- Hallucination Index: how often it outputs unsupported or fabricated content

In this benchmark, different LLMs make different “mistake trade-offs.”

Model-by-Model Performance: Who’s Accurate, Who’s Conservative, Who’s Risky?

Below is how the paper describes overall behavior across the five metrics (computed from the reviewers’ judgments). The headline results are surprisingly nuanced: no single model wins on everything.

Overall bests and trade-offs

• Perplexity and DeepSeek V3 achieved the highest overall accuracy (as stated in the paper’s abstract summary).
• But when you look at the detailed performance profile discussed in the results section, Gemini stands out with highest accuracy and recall and a relatively low hallucination index.
• Some models (like Falcon) are extremely conservative—great precision, near-zero hallucination—but they sacrifice coverage (recall).

Comparison table (from the study’s reported metric patterns)

A practical way to interpret this: models that feel the most “helpful” and detailed aren’t necessarily the safest. Meanwhile, the most conservative models may refuse to say things that could be true, leading to incomplete answers.

Why Prompt Complexity Triggers More Hallucination (and What “Tricky” Really Tests)

One of the most important findings is how strongly prompt difficulty affects error rates. The paper doesn’t just eyeball trends—it runs a statistical test.

Statistical result: prompt complexity matters a lot

They compute per-question error rates (defined as 1 - accuracy) across the four tiers and run a Kruskal–Wallis H-test:
- H(3) = 35.21, p < 0.001
- effect size: η² = 0.50

Translation: prompt complexity alone accounts for half of the observed variance in error rates. That’s huge.

Why the “Tricky” tier is different

The paper also reports something very telling from post-hoc comparisons:
- Tricky prompts differ significantly from Basic, Explanation, and Advanced
- But Basic vs Explanation vs Advanced aren’t significantly different from each other (pairwise tests with Bonferroni correction show p > 0.39 for those non-Tricky comparisons)

So Basic/Explanation/Advanced stress models in somewhat similar ways, but Tricky prompts demand a different kind of correctness: spotting an embedded false clause rather than trusting surface-level coherence.

The mean accuracy gradient across tiers (reported)

They provide per-model mean accuracy by tier (Basic has n=20, Explanation n=17, Advanced n=13, Tricky n=20). A few highlights from that table:
- Gemini: Basic 0.950, Explanation 1.000, Advanced 0.538, Tricky 1.000
- Falcon: Basic 0.650, Explanation 0.176, Advanced 0.385, Tricky 0.950
- Gemma: Basic 0.700, Explanation 0.412, Advanced 0.077, Tricky 1.000

Those numbers show an interesting effect: Tricky prompts sometimes produce higher accuracy when the model is forced into a binary “true/false identification” structure. The paper explains that this structure is handled more reliably than complex open-ended reasoning, because it gives the model less room to improvise.

Practical implication: if you’re designing prompts for research drafting, you can often reduce hallucination risk by structuring tasks like “verify X is true/false” or “identify the incorrect clause,” rather than asking for free-form narrative explanations.

The Key Trade-off: Completeness vs Hallucination Resistance (Precision–Recall Tension)

Here’s the big idea the paper keeps coming back to:

Models that output more complete, detailed responses often have a higher risk of unsupported claims.
And models that are very resistant to hallucination often do so by being conservative—meaning they may omit parts of the correct answer.

A concrete example from the results

• Falcon: precision 1.00 and HI 0.00%, but recall is 0.64 and RDI is 0.24
• Gemini and Claude 3.5 Sonnet: higher recall (coverage) and low hallucination compared to many others

This is why the paper argues against single-number rankings. If you only look at one metric, you can get fooled. For academic writing, you usually need both:
- enough coverage to be useful (recall/completeness)
- and low fabrication risk (precision/HI)

Response deviation (RDI) adds another layer

RDI isn’t “social bias.” It’s more like a calibration check: how far the model’s response behavior deviates from ground truth labels. The paper reports:
- lowest RDI around 0.01 for Grok and DeepSeek V3
- highest RDI of 0.24 for Falcon

So you could have a model that “behaves consistently” (low deviation) but still fabricates more than you’d like. That’s a subtle but important warning for researchers.

So Which Model Should You Choose for Academic Work?

The honest answer is: it depends what you need the model to do.

If you need the safest factual correctness

The paper’s findings suggest conservative models can be attractive:
- Falcon is highlighted as having zero hallucinations but low recall—great for “don’t invent,” not great for “write a full literature section.”

LLaMA 3 also looks conservative (high precision 0.94 and low HI 5.12%) but recall is lower (0.77). This can be useful when you want tighter factual handling and can tolerate omissions.

If you need completeness without going too risky

The paper frames Gemini and Claude 3.5 Sonnet as most robust overall:
- Gemini: 0.91 accuracy, 0.97 recall, 7.84% HI
- Claude 3.5 Sonnet: strong balance across all metrics, including low RDI = 0.02

If you rely on “helpful writing” you must add verification

Models like ChatGPT, Grok, and DeepSeek V3 are described as coherent and informative—but the paper reports higher hallucination indices around the 12% range for multiple systems (ChatGPT 12.00%, Grok 12.76%, DeepSeek V3 12.24%, Perplexity 12.00%).

And the paper explicitly notes that even Perplexity’s retrieval-augmented setup doesn’t fully eliminate hallucinations in sophisticated academic prompts. So retrieval helps, but it doesn’t make hallucination impossible.

A practical workflow suggestion (based on the benchmark logic)

If you’re using an LLM for academic writing, consider a two-step loop:
1. Ask for structured verification (especially for “Tricky-style” false-clause traps).
2. Then ask for drafting only after facts are verified—because hallucination risk rises when the model is free to generate more text.

If you want the benchmark framing again, this is exactly the kind of reliability gap discussed in the original paper.

Key Takeaways

• Fluency ≠ factual reliability. The paper shows several models can generate convincing academic text while still having relatively high Hallucination Index values (often around ~12% for some systems).
• Prompt complexity strongly drives errors. The study finds a statistically significant effect of difficulty on error rates (Kruskal–Wallis H(3) = 35.21, p < 0.001, with η² = 0.50).
• “Tricky” prompts behave differently. Embedded false clauses in plausible statements increase failure modes in a qualitatively distinct way; models often accept the compound statement rather than rejecting the false part.
• There’s a real precision–recall trade-off.
  • Falcon: HI 0.00% and precision 1.00, but recall 0.64 (it’s conservative enough to skip content).
  • Gemini/Claude 3.5 Sonnet: more balanced—higher recall and relatively low hallucination risk.
• You can’t trust a single metric. The paper uses five metrics (accuracy, precision, recall, RDI, HI) to avoid misleading rankings based on just one dimension.
• What to do next (for safer academic use): use structured verification prompts (true/false identification, clause-level checking) before drafting narrative text, and always treat LLM-generated citations/claims as needing verification.

If you’d like, I can also turn these findings into a simple “LLM trust checklist” for academic writing—like a mini rubric you can apply to your own prompts and outputs.