ChatGPT vs API Tests: What Safety Benchmarks Miss With Search

ChatGPT vs API Tests: What Safety Benchmarks Miss With Search
Modality, consistency, citations, abstentions, and safety blind spots

Introduction

If you’ve ever seen an AI safety leaderboard score and thought, “Okay, so this model is safe enough,” this new research should make you pause. A lot of LLM benchmarking—especially the kind used to argue models are reliable, safe, or ready to deploy—still boils down to one main idea: measure accuracy once, from one access method, under one condition.

But real deployments don’t work like that. Users interact through interfaces (like Chat UIs), systems may enable or disable web search, and the model may behave slightly differently each time it’s asked the same question. This is exactly what Ro Encarnación, Tina Behzad, Emma Lurie, and Danaé Metaxa investigate in their paper, based on new research from the original paper. They audit how results change when you vary modality (Chat UI vs API), web search on/off, and even repeat the same prompt multiple times.

The headline finding: if your benchmark only reports a single accuracy number, you may be missing important safety-relevant behaviors—like inconsistent answers, different wording and reasoning, different citations, and even inconsistent refusal (“abstention”). In their study, the differences across these dimensions weren’t rare side effects; they appeared across both of the benchmarks they tested.

Why This Matters

This is significant right now because AI governance is increasingly trying to turn evaluation into something procedural and auditable. Draft guidelines for benchmark evaluation (including automated ones) are emerging, and benchmarks are being treated like standardized “safety measurements.” The problem is: many benchmarks assume the measurement setup matches the real system users interact with. This paper shows that assumption can fail—sometimes in subtle ways accuracy alone won’t reveal.

Here’s a concrete scenario you could apply today: imagine a team using benchmark accuracy to greenlight an AI assistant for policy guidance or bias-sensitive content review. If they only test via API with web search disabled and single-run prompts, they might believe they’re seeing a stable “safety behavior.” But if the deployed experience is a chat UI with search enabled (common in consumer tools and internal copilots), the model’s accuracy can shift by up to ~8 percentage points, and more importantly, its response consistency, citations, and refusals can change. That means the tool might be “about right” on average while still being unreliable at the exact moments you care about—like edge cases, identity-related topics, or when the model needs to be consistently cautious.

This research also builds on a broader thread in AI evaluation: earlier work already questioned whether benchmarks translate to deployment reality. What feels new here is the granularity. Rather than only asking “Is it correct?” the authors measure how it answers and how that changes with operational conditions. They even show that repeatedly asking the same prompt can yield disagreements in a meaningful fraction of cases—up to 21% of prompts showing inconsistent selection of answers. That kind of instability is hard to treat as “measurement noise,” because it mirrors a real user experience problem: you can’t safely rely on a system that doesn’t hold still.

How the Audit Was Set Up: API vs Chat UI, Search On/Off, and Repeated Runs

The authors ran an “apples-to-apples” style audit using one model family (to reduce confounding) but multiple ways of accessing it. They define modality as the access method users experience: ChatGPT’s chat UI versus OpenAI’s API. They also treat web search as a capability condition and compare search enabled vs search disabled.

They test two widely used safety/bias benchmarks as case studies:
- BBQ (11 social bias categories; bias evaluation)
- SafetyBench (7 safety categories)

They used 401 prompts total, sampled with stratification:
- BBQ: 198 prompts (18 per category)
- SafetyBench: 203 prompts (29 per category)

Then they made the study much more “deployment-shaped” than typical benchmarking:
- 4 conditions: API/no-search, API/search, chat UI/no-search, chat UI/search
- 3 repeated runs per prompt per condition
- Total responses: 4,812 (198×4×3 + 203×4×3)

They used zero-shot prompting—no examples or extra instructions—to observe baseline behavior in each interface.

The Model Access Conditions Compared

Importantly, they did not try to force identical system prompts or interface behaviors between API and chat UI. Their goal wasn’t to “equalize everything,” but to audit what actually happens in deployed-like setups, including interface layers like chat-specific policies, formatting, and moderation.

This means their results are directly relevant to safety evaluation design—because it’s exactly these interface layers that change user-facing behavior.

Accuracy Isn’t Stable: Modality and Search Can Flip the Outcome Story

Let’s start with the metric benchmarks love most: accuracy—the proportion of responses matching the benchmark’s gold-standard answer. You might expect modality differences to be tiny. The study shows they are often small, but the “small” changes become safety-relevant once combined with other behavioral instability.

Accuracy changes across conditions (with real numbers)

When search is disabled:
- BBQ: chat UI 79.1% vs API 81.9% (≈ 2.8 pp gap)
- SafetyBench: chat UI 85.9% vs API 88.5% (≈ 2.6 pp gap)

When search is enabled:
- BBQ still slightly favors API:
- chat UI 75.9% vs API 77.8% (≈ 1.9 pp gap)
- SafetyBench flips:
- chat UI 84.4% vs API 80.6% (≈ 3.8 pp gap, opposite direction)

Search can reduce accuracy by up to ~8 points

The paper reports accuracy decreases with search enabled:
- Search reduced accuracy in both modalities, with the decrease larger for API than for chat UI (modality × search interaction reported as statistically significant).
- They describe the reversal on SafetyBench as strong enough to switch the modality advantage when search is on.

Even if your benchmark shows “the model is good,” it may be good only under the benchmark’s particular interface and particular retrieval condition.

And accuracy alone still doesn’t answer the most safety-relevant question: is the behavior stable and appropriately cautious when the same input gets repeated, reformatted, or retrieved differently? The authors go well beyond accuracy next.

What They Measured Beyond Accuracy: Consistency, Text Similarity, Citations, and Abstention

This is the core message you should carry forward: behavioral variation can be safety-relevant even when average accuracy looks similar.

The authors evaluate four additional dimensions:

1. Consistency: Do repeated runs pick the same answer?
2. Text similarity: Are responses phrased similarly (lexical overlap) and explained similarly (semantic similarity)?
3. Citation grounding (only in search-enabled conditions): Do citations overlap, and do modalities cite different sources for the same answer?
4. Abstention behavior: Does the model refuse the prompt consistently, or does it “flip” between refusal and answering?

Consistency: Up to 21% of prompts behave inconsistently

They define consistency as the share of prompts where all three runs in a condition select the same benchmark answer.

Key reported patterns:
- BBQ: inconsistency is higher in chat UI than API in both search settings, and increases with search.
- Example inconsistency rates:
- chat UI/no-search: 14.7%
- API/no-search: 13.1%
- chat UI/search: 21.2%
- API/search: 19.2%
- SafetyBench: modality effects reverse with search.
- without search:
- chat UI 12.3% inconsistency vs API 6.4%
- with search:
- API 12.8% vs chat UI 8.4%
- and one striking detail: SafetyBench/chat UI was the only combination where search decreased inconsistency.

Crucially, the paper also reports that repeated runs produced inconsistent responses in up to 21% of prompts. That’s not just “stochasticity”—it’s a user-facing reliability issue.

Disagreement shows modality effects beyond randomness

To check whether variability is just normal randomness, they compare:
- within-modality disagreement (API vs API repeated runs; chat vs chat repeated runs)
- between-modality disagreement (API vs chat UI for the same prompt)

Across conditions, they find between-modality disagreement is consistently higher than within-modality, ranging from 1.14 to 1.30× higher, and it widens with search.

They also report a model-level result:
- between-modality answer pairs have significantly higher odds of disagreement than within-modality pairs (reported as OR 1.51, CI [1.19, 1.92], p=0.002).

This suggests interface/access differences change behavior systematically, not just via randomness.

Response phrasing: APIs and chat UIs often talk differently, even when correct

They measure similarity in two ways:

• Lexical similarity (Jaccard overlap): how much word overlap happens
• Semantic similarity (embedding cosine similarity): how aligned the explanations are meaning-wise

Across conditions, within-modality responses are more similar than between-modality responses. For instance, in BBQ/no-search:
- within-modality Jaccard:
- API 0.59
- chat UI 0.40
- between-modality Jaccard: 0.15

So even if two modalities agree on the correct option, they may still differ heavily in how they explain it.

Citations: the two interfaces pull from different sources

Here’s one of the most policy-relevant findings: citation grounding differs substantially by modality, even when both are using web search.

They note:
- chat UI supports a broader citation display (including an added “More Results” panel), while API only includes citations in-text.

Reported citation patterns:
- BBQ:
- chat UI cites at least one source in 46.8%
- API cites in 32.2%
- SafetyBench (reversed):
- API cites at least one source in 73.9%
- chat UI cites in 60.6%

But the most revealing piece is overlap:
- At the URL level, overlap is tiny on both benchmarks—about 4% shared URLs (URL-level overlap reported as ~4% for BBQ and ~4% for SafetyBench).
- Domain-level overlap is still low: ~7% at the domain level (reported similarly for both benchmarks).

Even when both systems answer the same question correctly, they often cite different pages—or even don’t share any URLs on a large fraction of prompts (they report a substantial portion of prompts with no URL overlap).

Abstention: refusal behavior is inconsistent

Safety evaluations often treat refusal as a “safety feature.” But this paper shows abstention can be inconsistent across runs and interfaces.

They found:
- SafetyBench produced no abstentions in their experiment.
- BBQ had rare abstentions:
- six total across four distinct prompts in the no-search condition
- only one prompt triggered abstention in both modalities

Even more importantly, abstentions were not stable:
- For three of the four abstaining prompts, abstention happened on only one of the three runs
- The fourth happened on two of the three runs

That suggests abstention might behave like a brittle guardrail rather than a consistent safety mechanism—raising safety questions similar to: “If I ask again, do I get the restricted content?”

What This Means for AI Safety Evaluations: Design the Test Like the Deployment

So what do you do with this if you’re building, auditing, or governing AI systems?

1) Stop treating one accuracy number as “safety truth”

The paper’s findings directly challenge a common benchmark logic: if accuracy is high, reliability is high. Here, average accuracy is only part of the story. They show differences across modality/search that standard reporting hides:
- accuracy can change by several percentage points
- consistency can shift dramatically
- citations can come from entirely different sources
- abstention can flip across runs

A safety decision that ignores these dimensions is like a car safety rating based on a single test drive under one set of conditions, reported as one averaged outcome.

2) Evaluate modality and search jointly (not separately)

The biggest trap in many evaluations is assuming “same model = same behavior.” This audit shows that’s not reliably true across:
- chat UI vs API
- search enabled vs disabled

And sometimes the direction of effects changes by benchmark (the SafetyBench reversal under search is a great example).

3) Include repeated-run measures for behavioral stability

If a system can pick different answers for the same prompt in up to 21% of prompts, then “single-run” benchmarking is not enough to characterize reliability.

For safety, that matters because risk doesn’t only come from average-case performance—it comes from how systems behave when something goes wrong or when users re-ask, rephrase, or re-trigger the same behavior.

4) Treat citation and abstention behavior as first-class safety metrics

Citation overlap at the URL level being around 4% is a sign that grounding is not an incidental detail. In safety contexts, users may trust (or contest) answers based on where they come from and whether sources are consistent.

Likewise, inconsistent refusal behavior means guardrails may not be reliably enforced in the user’s lived interaction loop.

5) Acknowledge evaluation constraints: chat UI auditing is harder—but still needed

The authors also discuss a structural barrier: evaluating chat UIs costs more (session management, infrastructure, anti-abuse constraints), while API auditing is cheaper and more scalable. That naturally biases the benchmark ecosystem toward API-only results—turning “easy to measure” into “good enough to assume.”

If evaluation systems keep measuring what’s easiest instead of what’s most representative, the literature will systematically under-measure the safety-relevant behaviors users actually experience.

If you want the full methodology and conditions, revisit the original paper—it’s especially useful for anyone designing evaluation protocols.

Key Takeaways

• Accuracy-only benchmarking can hide safety-relevant variation. Even when accuracy differences are modest, other behaviors shift substantially.
• Modality matters. On search-disabled conditions, chat UI was less accurate than API on both benchmarks (e.g., BBQ 79.1% vs 81.9%, SafetyBench 85.9% vs 88.5%).
• Search can flip results. On SafetyBench with search enabled, chat UI became more accurate than API (84.4% vs 80.6%).
• Consistency is often worse than you think. Repeated runs produced inconsistent answers in up to 21% of prompts, and inconsistency changed with both modality and search.
• Responses aren’t just “different ways of saying the same thing.” Text phrasing and semantic explanation can diverge strongly between API and chat UI even when answers match.
• Citations are modality-dependent. URL overlap between modalities was about 4% on both benchmarks when search was enabled, meaning grounding differs even for the same prompt.
• Abstention/refusal can be unstable. BBQ showed abstentions that appeared in only some runs, suggesting guardrails may not behave consistently across repeated user interactions.
• For safety evaluations: measure modality + search conditions, include repeated runs, and report more than accuracy—at least consistency, citation grounding, and abstention behavior.