ChatGPT Names Brands It Already Knows, Graphite Study Finds

Brands named on 15% of ChatGPT's cited pages still saw just 11% median visibility with search if the model rarely named them without search, Graphite found.

Wire notes

  • Median visibility with search was 11% for brands named in under 25% of no-search answers despite 15%+ cited-page presence, per Graphite's Oct. 16 analysis
  • 59.6% of ~957,000 ChatGPT fan-out queries contained a brand name; 22.7% used a site: operator
  • Logged-out ChatGPT citation rates rose from 56.6% (week of Aug. 3) to 88.5% (week of Sept. 28)
  • Ahrefs found 14.4% of 66.6 million logged-out ChatGPT responses in August-September included fan-out queries
  • Models released since July show a two-to-five-month gap between knowledge cutoff and release
Some Brands On ChatGPT’s Cited Pages Still Got Few Mentions via @sejournal, @MattGSouthern
PhotoSome Brands On ChatGPT’s Cited Pages Still Got Few Mentions via @sejournal, @MattGSouthern — Emiliano Vittoriosi / unsplash

Brands that ChatGPT rarely named without search stayed rare even when ChatGPT used search and those brands appeared on at least 15% of the pages it cited. That is the headline result from an Oct. 16 analysis by Graphite, a growth agency that sells AEO services, which compared logged-out ChatGPT answers with search switched on and off.

Among brands substantially mentioned on at least 15% of cited pages, median visibility with search was just 11% when ChatGPT named them in under a quarter of its no-search answers. Graphite argues brands should work on what AI models learn during training, not only on citations, and calls that approach parametric knowledge optimization, or PKO.

How Graphite ran the comparison

The study uses entity-comparison prompts — questions asking ChatGPT to compare brands or products — drawn from Graphite's Aug. 20 paper on prompt tracking. It covers 3,115 entity-question pairs across 57 questions, collected in August from logged-out ChatGPT. Visibility means the share of responses that mention a brand or product.

For search-based answers, Graphite also measured the share of cited pages that substantially mentioned each brand, then split brands into three groups by that share. In all three groups, brands named more often without search were named more often with search. Visibility via search rose alongside the share of cited pages mentioning the brand, which Graphite says suggests both factors matter.

Graphite states the same relationship held, with comparable strength, in two replications:

  • 100 prompts run through the API with search explicitly enabled and disabled
  • 200 prompts tracked through Graphite's tracking tool over a different three-week window

One caveat: the Oct. 16 write-up doesn't explain how Graphite separated non-search responses in the main comparison, and only the 100-prompt API replication is described as explicitly disabling search. The write-up also lacks controls for each brand's degree of general awareness.

What does ChatGPT search for before answering?

Graphite counted brand names in roughly 957,000 ChatGPT fan-out queries — the background searches it runs before answering — from 22,953 non-branded entity-comparison prompts collected between August and early October. None of the prompts named a brand, yet 59.6% of the fan-out queries contained one, as labeled by Anthropic's Claude Haiku 4.5 model, and 22.7% used a site: operator. Graphite says a brand appearing in a fan-out query raises the chance that pages featuring that brand get retrieved.

Suganthan Mohanadasan found a similar pattern in a smaller test he wrote about for SEJ in August: in 21 of 27 conversations on a single logged-in account, ChatGPT's first search query included brand names he hadn't typed.

How often does ChatGPT cite sources?

Between two weeks of fixed entity-comparison prompts in Graphite's tracking tool, logged-out ChatGPT responses with citations rose from 56.6% in the week of Aug. 3 to 88.5% in the week of Sept. 28. Logged-out Gemini moved from 79.0% to 81.2% over the same weeks. Graphite says the increase suggests more frequent search usage, and notes the percentages could go down.

A citation doesn't prove a search happened. In an earlier paper, Graphite noted that as of May, ChatGPT could add inline links without displaying any search evidence. Ahrefs, in its Oct. 12 fan-out report, found 14.4% of 66.6 million free, logged-out ChatGPT responses in August and September included fan-out queries — but the two datasets don't map directly, since Ahrefs counted general prompts while Graphite counted citations on comparison prompts.

Training cutoffs and what brands can measure

Graphite calculated the gap between each model's stated knowledge cutoff and its release date, using Anthropic's training data cut-off for comparability across all three vendors. For models released since July, the gap ran roughly two to five months — what Graphite calls "the minimum time-to-impact for newly published content." The measure doesn't capture when an article is crawled into a training set or whether answers change.

Graphite proposes two exploratory methods:

  • Check brand visibility via the API with search turned off whenever a new model ships, collecting at least 50 responses per prompt
  • Allow training crawler access to a website, and check for crawler restrictions from CDNs or firewalls

Google's crawler page says its Google-Extended robots.txt token controls whether crawled content trains future Gemini models and grounds Gemini Apps and Vertex AI grounding — but doesn't affect Google Search inclusion or rankings.

The data shows visibility with search rose alongside both no-search visibility and cited-page mentions, but it doesn't prove that increasing either will make ChatGPT name a brand more often. Graphite says brands should expect the two-to-five-month cutoff lag before new content reaches a model, and because AI labs don't disclose training data, no one knows which pages made the cut. With ChatGPT's citation rate climbing from early August through late September, search-on tracking will need repeating too.

via graphite.io (Original)

Filed under

Share this article:

More from James Calloway

James Calloway

Show full bio

Market editor covering technology and product news in marketing, advertising, marketing technology, media, SEO and search at Marketing Herald.

175 articles

More on the wire

« Previous articleNext article »