Where ChatGPT's Citations Come From, and Why Google Rankings and the API Only Partly Predict Them

If you want to know whether ChatGPT cites your website, there are two shortcuts most teams reach for. The first is to check where you rank on Google. The second is to ask OpenAI's API the same question and see which pages it cites. A new study tested both shortcuts against what ChatGPT actually did, and neither one came close to matching it page for page.

The study is a preprint posted to arXiv on October 11, 2026, by Yunxuan Jiang, Shiqi Zhang, Anqi Xing, Xiangyu Chang and Xiaokui Xiao.¹ A preprint is a paper shared before other researchers have formally reviewed it, so treat the results as early evidence, not settled fact. Search Engine Journal covered it on October 18.² Below, we walk through what the researchers measured, what they found, and what it means if your job is to track AI search visibility.

How ChatGPT search finds sources in the first place

Before looking at the study, it helps to know the basic steps ChatGPT follows when it searches the web. OpenAI's own help page describes them.³

1. It decides whether to search. ChatGPT may search automatically when a question would benefit from current information. Users can also start a search by hand.
2. It rewrites your question. OpenAI says ChatGPT usually turns your question into one or more targeted search queries. It sends those to search providers, and it may send follow-up queries after it reads the first results.
3. It uses outside search partners. OpenAI says ChatGPT search "sometimes partners with other search providers," and it points to Microsoft's and Shopify's privacy policies as examples.
4. It ranks and cites. OpenAI says ChatGPT ranks results using multiple factors and that placement is not guaranteed. Answers that use search may include citations, which are clickable links to the source.

There is also a crawler involved. A crawler is a program that visits web pages automatically. OpenAI's crawler for search is called OAI-SearchBot. OpenAI's crawler page says sites that block OAI-SearchBot "will not be shown in ChatGPT search answers," although they can still appear as plain navigation links.⁴ That page lists two other agents. GPTBot collects content to train OpenAI's models. ChatGPT-User visits a page when a user's request calls for it, and OpenAI says it does not crawl the web automatically.

So the first, very practical check is simple. If your robots.txt file blocks OAI-SearchBot, nothing else in this article matters much. OpenAI says changes to robots.txt can take about 24 hours to take effect for search.⁴

The step that matters most for this study is step 2. ChatGPT does not search for your exact question. It searches for its own rewritten version. We explained this process, often called query fan-out, in our guide to how query fan-out works in AI search.

What the researchers measured

The team recorded what happened inside real ChatGPT web sessions. They captured the data stream the ChatGPT website sends back to the browser. From it, they pulled out four things for each answer: the search queries ChatGPT actually ran, the pages it retrieved, the pages it used as support, and the final citations.¹ The paper says these came from the authors' own logged-in ChatGPT accounts.

Here is the setup in plain numbers.

For each question, the ChatGPT runs, the API calls and the Google results were collected within the same three-day window.¹ That matters because it means the three sources were compared at roughly the same moment, not weeks apart.

The researchers compared matches at three levels:

• URL level: the exact same page.
• Hostname level: the same site. The paper keeps subdomains separate, so blog.example.com and example.com count as different sites.
• Entity level: the same named things, such as brands, products or organizations, picked out of each result's title and short description.

ChatGPT almost never searches your exact question

The first finding is about query rewriting. In the runs where the researchers could see the queries, GPT-5.3 Instant, GPT-5.4 Thinking and GPT-5.5 Instant never once searched the exact wording of the original question.¹ GPT-5.2 Instant did it in only 5.5% of runs.

How much each model searched also varied a lot.

Source: Jiang et al., Table 3 and Section 3.¹

So GPT-5.3 Instant and GPT-5.5 Instant usually search once and stop. GPT-5.2 Instant always ran more than one search, but almost always in a single round. The Thinking setting searched almost 12 times per answer on average, across more than three rounds. In other words, the number of chances your page has to be found depends heavily on which model is answering.

One small detail is worth knowing. For GPT-5.3 Instant, the queries were visible in only 1,339 of the 1,600 runs.¹ The rest did not show what was searched. So some query statistics are based on the runs where search was visible.

Chinese questions, English searches

The language finding is striking. When the question was in Chinese, GPT-5.3 Instant searched only in English in 67.0% of runs. GPT-5.5 Instant did the same in 69.6% of runs.¹ GPT-5.2 Instant behaved differently: 96.8% of its runs used both Chinese and English queries.

This means a page written only in the user's language can miss out, even when the question was asked in that language. The study only tested Chinese and English, so we can't say how other languages behave. But if you sell in non-English markets, it's a pattern worth checking in your own data.

The pages ChatGPT finds change a lot from run to run

The researchers asked the same question 20 times and looked at which pages ChatGPT retrieved each time. Retrieved means ChatGPT pulled the page up as a possible source. It does not mean the page was cited.

Source: Jiang et al., Section 3.3.¹ The overlap score (called Jaccard similarity) runs from 0, meaning no pages in common, to 1, meaning identical lists.

Put simply, almost half of the pages GPT-5.5 Instant found appeared in just one run out of 20. Fewer than one in ten were found reliably. That's why a single check of a single prompt can give a misleading answer, which is the same issue we covered in how many checks you need before trusting an AI visibility number.

Notice the overlap scores too. Two runs share more at the site level than at the page level. The same sites keep coming back, even when the exact pages change. That pattern shows up again in the next section.

Pages that keep coming back are cited more often

The researchers also checked whether a page that shows up in many runs is more likely to be cited. They hid one run, counted how often each page appeared in the other 19, and then checked whether that count predicted citation in the hidden run.

It did, but only modestly. The paper reports this using a score called AUC. An AUC of 0.5 means the guess is no better than a coin flip. An AUC of 1.0 means a perfect guess.

Source: Jiang et al., Figure 3.¹

The authors are careful here, and we should be too. They call this observational. It does not show that getting retrieved more often will cause you to be cited more.¹ It only shows the two tend to go together.

They found one more pattern. Of the short text snippets attached to cited pages, 16.5% (GPT-5.3) and 16.9% (GPT-5.5) were at least 500 characters long. For pages that were not cited, only 1.5% and 1.9% were that long.¹ The authors treat snippet length as a signal they could observe, not as a cause. So this is not proof that longer passages win citations.

Google and the API only partly predict ChatGPT's citations

This is the headline result. The researchers took every page ChatGPT cited across the 20 runs of each question. Then they asked how many of those cited pages also appeared in Google's results or in the API's list of sources.

A quick note on the API. OpenAI's web search tool can return two lists.⁵ One is the citations shown in the answer. The other is a full "sources" list of every URL the model consulted, which OpenAI says is usually longer than the citations. The study tested both. "API sources" below means that longer list.

Source: Jiang et al., Tables 4 and 5.¹

Here is what that table says in plain words:

• Google alone is a weak guide at the page level. Fewer than 1 in 10 pages ChatGPT cited showed up in Google's top five pages of results for the same question.
• The API does better, but still misses most pages. Its full sources list caught about 23% of the pages ChatGPT cited.
• Even both together miss nearly three in four cited pages. Combined, they found 28.3% (GPT-5.3) and 26.2% (GPT-5.5).
• At the site level, the picture roughly doubles. The combined shortcuts found about half of the sites ChatGPT cited.

The match also works the other way. Of the pages the API cited in its own answers, only 16.4% were also cited by GPT-5.3 Instant, and 23.3% by GPT-5.5 Instant.¹ Agreement was higher for GPT-5.5 Instant, but most API citations still didn't show up in ChatGPT's.

Part of the gap with Google makes sense once you remember step 2 from earlier. Google was given the original question. ChatGPT searched a rewritten version, often in a different language, through its own search partners. They were never really looking at the same results page.

Can a model predict which pages get cited?

The researchers also built a prediction model. It used only signals from Google and the API, never anything from ChatGPT itself, to guess which pages ChatGPT would cite. It scored an AUC of 0.782 at the page level for GPT-5.3 Instant and 0.779 for GPT-5.5 Instant. At the site level it scored 0.763 for both.¹

That sounds solid, but there's a catch. The model could only rank pages that Google or the API had already found. Cited pages outside that pool were left out of the test.¹ And as the table above shows, that pool missed most cited pages. So the model is good at sorting the pages it can see, but it can't see most of them.

What this means for how you measure AI visibility

This is where the study becomes useful for anyone tracking how often AI tools mention or cite their brand. The authors' own advice is that teams should separate "retrieval visibility" from "citation visibility."¹ Retrieval visibility means ChatGPT found your page. Citation visibility means it actually linked to it in the answer. Being found does not guarantee being cited.

Below is our view of what each common measuring method can and can't tell you, based on the study's results.

Match rates are rounded from Jiang et al. Tables 4 and 5, covering GPT-5.3 Instant and GPT-5.5 Instant.¹

None of this means Google rank or the API are useless. They are cheap and fast. But they are proxies, meaning stand-ins for the real thing. If they are your only measure, you could miss most of the pages that actually get cited.

Worked example: why page-level and site-level matching tell different stories

This example is illustrative. The brand, sites and numbers are invented to show the math. They are not from the study or from a real client.

Say you run a hiking gear brand called Ridgeway Gear. You track one prompt: "best waterproof hiking boots for beginners." You run it 20 times in the ChatGPT app and also run it 20 times through the API.

Across the 20 app runs, ChatGPT cites 20 different pages. Those pages come from 8 different sites. Your site, ridgewaygear.com, is one of them. Three of your pages got cited at some point: a buying guide, a sizing page, and a product page.

Now compare the API's sources list against what the app cited:

• The API's list includes 5 of the 20 cited pages.
• The API's list includes at least one page from 4 of the 8 cited sites.
• From your site, the API found your blog review of trail shoes. The app never cited that page.

Here is the math:

• Page-level match: 5 ÷ 20 = 25%
• Site-level match: 4 ÷ 8 = 50%
• Your brand at page level: 0 of your 3 cited pages matched = 0%
• Your brand at site level: your site appears in both = match

So you could reach two very different conclusions from the same data:

1. If you only look at exact pages, the API says ChatGPT ignores you. That would be wrong. ChatGPT cited three of your pages.
2. If you only look at sites, the API says you're in. That's true, but it hides which pages are doing the work. The trail shoe review the API found was never cited.

The useful answer needs both levels. The site level tells you whether your brand is in the conversation. The page level tells you which content to improve or protect. We picked 25% and 50% on purpose. They sit close to the study's combined Google plus API figures, about 26% to 28% of cited pages and about half of cited sites, so the example reflects the gap the researchers found.

One more trap from the paper's method: it treats subdomains as separate sites.¹ If your blog lives on blog.ridgewaygear.com and your shop on ridgewaygear.com, a site-level count may split them. Decide up front whether you want to group them, and stick with that choice.

A measurement checklist based on the study

• Run each prompt many times. Almost half of retrieved pages showed up in only 1 of 20 runs. One check is a sample of one.
• Track both page level and site level. Site-level matches were roughly twice as common as page-level matches. Report both, and say which one a number refers to.
• Log the model version for every answer. Search behavior changed sharply between GPT-5.2, 5.3, 5.4 Thinking and 5.5. The researchers also saw two sessions quietly switch to a fallback model after 14 and 16 runs.¹ If you don't log the model, you might be mixing two different systems in one chart.
• Capture the rewritten queries, not just your prompt. ChatGPT's newer settings never searched the exact original wording. The queries it really ran tell you which topics, and which language, you need to show up for.
• Separate retrieval visibility from citation visibility. Count how often you were found, and separately count how often you were cited. A page that is found but rarely cited needs different work from a page that is never found.
• Check that OAI-SearchBot can reach your site. OpenAI says blocked sites won't appear in ChatGPT search answers.⁴
• Re-test when the model changes. The authors say strategies should be re-evaluated when the model or serving setup changes.¹

Limits of the study

This is careful work, but it has real limits. Keep these in mind before you quote it.

• Not peer reviewed. It is an arXiv preprint. arXiv does not peer review papers.²
• Small question set. 80 questions in two languages. The results may not hold for other topics, industries or languages.
• Older models. The data ran from March 29 to July 29, 2026. Search Engine Journal notes that every setting in the paper predates GPT-5.6 and GPT-6.² Newer models may search and cite differently.
• Observational, not causal. The authors say their results "do not imply that publishing more pages or making a source appear more often in retrieval will increase its likelihood of being cited."¹
• Some settings undefined. The paper does not state the location, language or device settings for its Google results, or the region of its ChatGPT accounts. The ChatGPT data came from the authors' own logged-in accounts.
• Partial query visibility. For GPT-5.3 Instant, executed queries were missing for 261 of the 1,600 runs.¹
• Entities were extracted with AI help. Named entities were picked out partly by GPT-4.1-mini, with manual checks. Brands were folded into the entity group, so the paper makes no separate claims about brands.¹

The takeaway

ChatGPT doesn't search your question. It searches its own rewritten version, sometimes in a different language, and the pages it finds change from one run to the next. That's why Google rank and the API only partly predict what it cites. They recovered about a quarter of the exact pages and about half of the sites.

So if you want to know what ChatGPT says about your brand, the most reliable method is to ask ChatGPT itself, many times, and record what it does. Use Google and the API as quick, cheap background signals, not as the final answer.

See which of your pages ChatGPT actually cites

Ripplix tracks real AI answers, so you can see which of your pages and sites get cited and which competitors show up instead. Start with a free report on your own brand.

Get your free AI Visibility Report →