ChatGPT Citations Aren’t Just “Ranking”: New GEO Clues (2026 Study)

ChatGPT Citations Aren’t Just “Ranking”: New GEO Clues (2026 Study)

Introduction: What a citation really means in ChatGPT Web

If you’ve ever tried to figure out why one webpage shows up as a citation in ChatGPT (and another similar page doesn’t), you’re not alone. “Citations” in ChatGPT Web aren’t the same thing as search visibility in the classic sense. A system can retrieve lots of pages, but only some become the citations you actually see—and that extra “citation selection” step can make or break visibility.

New research from this arXiv paper digs into the mechanics behind ChatGPT Web citations. Instead of just analyzing the final answers (like which pages were cited), the authors observe the internal execution flow: how ChatGPT rewrites your query, runs web retrieval, and then chooses which references make it into the final citation list.

Even better, the study compares multiple production settings—covering Instant and Thinking modes across ChatGPT 5.2 to 5.5—to see how those behaviors change with the model setup. The headline finding: candidates that show up more often across repeated runs are more likely to be cited, but the relationship is stronger at the hostname and entity levels than at exact URL matching.

Why This Matters: GEO teams need to optimize for “citation selection,” not just “found pages”

Here’s the part that feels immediately relevant right now: a lot of generative engine optimization (GEO) advice implicitly treats citation outcomes like an extension of SEO—more exposure → more likely citation. This study shows that’s only half the story.

Why it matters now: model behavior (especially query rewriting and multi-round retrieval) changes across settings, and production systems can behave differently even for the “same” user-facing mode. So if you optimize using one measurement method (like “what Google ranks”), you may be optimizing the wrong stage of the funnel.

A scenario you could apply TODAY: imagine you run a customer-support knowledge base site. You want specific articles to appear as citations when users ask questions like “How do I reset my device?” or “What’s the warranty policy for X?” Traditional SEO might push you toward winning particular rankings. But this paper suggests you should also track whether your domains (hostnames) and content entities appear consistently in retrieval, because citation selection correlates more reliably at those levels than at exact URL matches. In practice: you’d monitor not only which pages Google surfaces, but whether ChatGPT’s web tool frequently retrieves your site and attaches the relevant named entities (product names, brand terms, services) before citation happens.

How this builds on earlier AI search research: previous work often compared citation outputs across systems or looked at which sources were cited in the final response. This paper adds a crucial missing layer: it observes the retrieval-to-citation funnel using server-sent event (SSE) traces, then tests external proxies (Google Search and the OpenAI Responses API) against those internal behaviors. The result is a more actionable GEO picture: there’s retrieval visibility, and there’s citation visibility—and they don’t always line up.

What the researchers actually measured inside ChatGPT Web (and why SSE traces matter)

This study is built around a pretty practical idea: if citations are the end result, the real work happens earlier. The authors treat a ChatGPT Web run as a pipeline with three components:

1. Query rewriting: the system reformulates the user query into executed web-search queries.
2. Web retrieval: it returns candidate webpages (including “supporting” pages attached to referenced candidates).
3. Answer synthesis: it selects which references (and attached supporting pages) become the final citations.

The key measurement advantage is that they use SSE traces to observe what happened during the run—not just what was shown after the fact. That lets them label each candidate webpage as either “cited” (y_W=1) or “not cited” (y_W=0) based on the final citation list.

Their dataset: aligned queries + multiple settings + repeated runs

The authors grounded queries in real user behavior by sampling from Search Arena, a crowd-sourced dataset of 24k in-the-wild search-augmented LLM interactions. They selected 80 queries across four intent types and two languages:

• Factual Lookup
• Information Synthesis
• Analysis
• Recommendation

They excluded other intent types because they rely more on parametric knowledge or target a destination rather than cited answers.

Then they collected SSE traces for four ChatGPT Web settings:

• chatgpt-5.2-instant
• chatgpt-5.3-instant
• chatgpt-5.4-thinking
• chatgpt-5.5-instant

For the aligned comparison sets:
- chatgpt-5.3-instant and chatgpt-5.5-instant: 80 queries, each with 20 independent runs per query
- chatgpt-5.2-instant and chatgpt-5.4-thinking: smaller query sets due to deprecation timing, with fewer runs per query

So the study isn’t just a single-pass snapshot. It’s designed to test how run-to-run variation affects what gets cited.

Instant vs Thinking: how query rewriting and retrieval rounds change the citation funnel

A big theme is that different ChatGPT Web settings behave differently before the citation stage. The paper distinguishes:

• A run = one complete ChatGPT Web response
• A retrieval round = one use of the web search tool
• An executed query = one query issued within a retrieval round

Comparison across settings (what changes where)

Here’s what the authors observed for Instant vs Thinking in terms of query rewriting and retrieval rounds:

Two practical takeaways jump out:

1. Instant is usually single-round, but that doesn’t mean it’s behaviorally identical. chatgpt-5.5-instant issues multiple queries much more often than 5.3-instant (31.3% vs 6.2%).
2. Thinking is where you see multi-round retrieval and narrowing. In chatgpt-5.4-thinking, multi-round retrieval happens in ~74% to 88% of runs depending on intent, and later rounds shift toward more targeted query patterns (e.g., increasing site: usage from 25.1% in first round to 54.7% later; quoted phrases from 1.0% to 22.7%).

This matters for GEO because if your optimization assumptions depend on “how the user query gets searched,” those assumptions can break across model settings.

Why “cited” correlates with recurrence (and why hostname/entity beats exact URL)

One of the most actionable findings is about recurrence across repeated runs. The authors found that candidates appearing more often across runs are more likely to end up cited.

Recurrence prediction (leave-one-run-out) shows a strong link

To avoid unfair overlap (“if it appears often, it’s more likely labeled positive in the same runs”), they use a leave-one-run-out setup:

• For each candidate in a held-out run, measure how often it appeared in the other 19 runs.
• Use that recurrence to predict whether it gets cited in the held-out run.

The reported AUC results (higher = better prediction) show the pattern clearly:

• chatgpt-5.3-instant
  • URL AUC: 0.619
  • Hostname AUC: 0.666
  • Entity AUC: 0.688
• chatgpt-5.5-instant
  • URL AUC: 0.651
  • Hostname AUC: 0.709
  • Entity AUC: 0.711

The authors also note recurrence is more informative at hostname and entity levels than at exact URL level. That matches what they observe about stability: URLs churn more, while hostnames/entities remain more consistent across runs.

Candidates churn at URL level—but stabilize at coarser levels

They measure stability using:
- “one-off” candidates (appear in only one run)
- “stable core” (appear in at least 14 runs)
- pairwise Jaccard similarity between runs

At the URL level:
- chatgpt-5.3-instant: 40.8% one-off URLs; 9.4% stable core
- chatgpt-5.5-instant: 44.5% one-off URLs; 7.7% stable core

At the hostname level:
- Jaccard similarity rises to about 0.408 and 0.402 respectively.

At the entity level:
- Jaccard similarity is around 0.421 and 0.404.

Translation into human terms: across repeated attempts, ChatGPT may pull different specific pages from the same domain, or different pages that still mention the same key entities. Citations seem to care more about that “domain/entity footprint” than about the exact URL string.

Snippet length helps—but it’s weaker than recurrence

They also test snippet length (how long the retrieved excerpt is). Snippet length has some predictive power:

• AUC ~0.612 (5.3-instant) and 0.625 (5.5-instant)

But it’s weaker than recurrence. Long snippets correlate with citation more often in the tail: about 16.5%–16.9% of cited snippets are at least 500 characters, versus only 1.5%–1.9% of uncited snippets.

For GEO, the practical implication is subtle: snippets may reflect how “self-contained” a source is once retrieved, but snippet length isn’t a magic lever by itself. It can be confounded by page type, formatting, or retrieval position.

How well can external proxies mimic citation behavior? Google vs the Responses API

A major part of this paper isn’t just “what ChatGPT does,” but “what we can measure externally.” The authors test two proxies:

• Google Search (via SerpAPI): top 5 organic results per query
• OpenAI Responses API: web-enabled calls to gpt-5.3-chat-latest and gpt-5.5, with reasoning.effort="none" for gpt-5.5 to better match Instant-like behavior

They then evaluate proxies on three levels:
- exact URL
- hostname
- entity

Proxy trade-off: candidates vs citations

The responses API naturally gives two useful signals:
- candidate exposure (pages returned in search results)
- citation extraction (pages the model directly cites in the API output)

The paper finds a consistent pattern:
- API candidates → higher recall (find more of what ChatGPT cites, but less precisely ranked)
- API citations → higher precision (closer alignment with what ChatGPT Web actually selects)

Proxies match hostname/entity better than exact URLs

One of the most important external measurement findings:

External proxies recover far fewer exact cited URLs than cited hostnames.

For chatgpt-5.3-instant / chatgpt-5.5-instant:
- cited URLs recovered: 28.3% / 26.2%
- cited hostnames recovered: 50.3% / 48.7%

That gap is the big reason GEO dashboards that focus only on exact URLs can be misleading. If your measurement is too granular, proxy coverage will cap how well you can predict citation outcomes.

Combining proxies improves prediction—but candidate coverage still limits the ceiling

They even combine signals from Google and the Responses API inside a supervised XGBoost model. At the “conditional” level (meaning: only among candidates discovered by proxies), the results are promising:

AUCs around:
- 0.782 / 0.763 / 0.743 for chatgpt-5.3-instant (URL / hostname / entity)
- 0.779 / 0.763 / 0.742 for chatgpt-5.5-instant

But there’s a hard constraint: proxies don’t discover all candidates. Even with combined external signals, the system only recovers about one quarter of cited URLs.

If you’re using external monitoring, this is a “don’t overpromise” moment: you can rank within the candidate set, but you can’t guarantee discovery of the exact citations.

What GEO should do differently: retrieval visibility vs citation visibility, and monitoring at the right granularity

The paper ends with GEO implications, and they’re more nuanced than “get ranked higher.”

1) Separate “retrieval visibility” from “citation visibility”

Their proxy evaluation basically shows two stages can diverge:
- Google / Responses API “exposed” pages
- ChatGPT Web “selected” references for citations

GEO should treat those as different objectives. Otherwise, you might optimize for retrieval exposure and still miss citation selection.

2) Monitor executed queries and run-to-run variation

ChatGPT rewrites queries. For example:
- exact-match to the user query appears in only 5.5% of 5.2-instant runs and in none for 5.3-instant / 5.5-instant (observed alignment set)

Also, repeated runs retrieve different pages. Since “more frequent across runs” predicts citation better, single measurements can be misleading.

Practical action:
- Run repeated probes for the same query (within reasonable limits)
- Track which executed queries get used, not only the user-facing query text

3) Track hostname and entity visibility, not only exact URL

Because stability is higher at coarse levels and proxies match there better, GEO monitoring should include:
- hostname-level presence
- entity-level presence (brands/products/names)

The recurrence → citation effect is stronger at those levels than exact URL level. So if your content strategy is domain-wide (or entity-wide), you’ll get better signal than chasing one specific permalink.

4) Re-evaluate when model settings or serving environments change

This is the “operations” lesson. The paper shows predictive signal importance shifts across 5.3-instant vs 5.5-instant. So GEO tactics that worked under one setting might not transfer cleanly.

If your serving environment changes (model version, reasoning effort, tool behavior), you should re-run your measurement calibration—especially the importance weights you might be using in a decision model.

Key Takeaways

• ChatGPT citations aren’t just search ranking. There’s a full retrieval-to-citation funnel inside ChatGPT Web.
• Candidates that recur across repeated runs are more likely to be cited. Recurrence predicts citation with AUC up to about 0.711 at the entity level.
• Exact URLs are unstable; hostnames and entities are more stable. Citations correlate more strongly with hostname/entity presence than with exact URL matching.
• Instant vs Thinking changes the pipeline. Thinking often does multi-round retrieval and narrows targets later; even Instant differs across 5.3 vs 5.5 in query behavior.
• External proxies help, but can’t fully replace internal signals. Google + the OpenAI Responses API recover only about 26%–28% of cited URLs, though they do better on hostnames (~49%–50%).
• For GEO monitoring, optimize your measurement granularity. Track retrieval visibility vs citation visibility separately, and monitor at hostname/entity levels in addition to URLs.
• Recalibrate when the deployed model setting changes. Predictive signals shift across settings, so “set and forget” GEO measurement can drift.

If you want, tell me what kind of site you’re optimizing (news, docs, ecommerce, SaaS support, etc.) and what citation use-case you care about—I can suggest a practical monitoring checklist aligned with the hostname/entity-first strategy this study supports.