AI Search Tools Cited a Fabricated Concept Within a Week, Study Finds

By Tom Whitfield

Eight of 10 AI search tools cited a made-up concept within seven days of researchers publishing posts about it, according to a paper from Fudan University posted on arXiv on Oct. 8. ChatGPT, Perplexity, Grok and Google AI — the four English-web tools tested — all picked up the fabricated "Cactus Gravity Protocol" within the week.

The team, led from Fudan, published posts from ordinary accounts in March and April on nine sites the tools already cited often, including Medium, YouTube and Sohu. On two other topics — an opinion question with existing discussion and a fictional keyboard — the same four tools picked up almost nothing in the same period.

The paper uses the label "Google AI" and links to Google's general AI site, but does not specify which Google product was tested.

How the test worked

In March, the researchers ran 120 questions through 10 AI search tools and recorded every site each tool cited. Reddit ranked as the most-cited site for Google AI and Grok, YouTube for Perplexity, and Wikipedia for ChatGPT. Six to 10 of each tool's 10 most-cited sites accepted posts from ordinary users.

April tests of 22 posting platforms rated Medium, GitHub, LinkedIn and Facebook easy to join and post on; Reddit's karma requirements and strict deletions made posting harder. For each tool, the team picked its most-cited site open to new accounts, yielding the nine test sites.

The team posted articles on three topics:

• the fictional "Cactus Gravity Protocol"
• whether AI would supplant junior programmers within five years
• a fictional keyboard

ChatGPT drafted each article; the researchers then manually revised them and planted fabricated names and numbers to track repetition. Pre-posting checks confirmed no tool gave informative answers on the fictional topics. After posting, the researchers ran the same five questions per topic on every tool daily for one week.

Unlike a Cornell Tech paper from June that tested open-source research agents in simulation, this team posted on the live web and queried the tools through their public interfaces.

What did the four English-web tools cite?

Counting answers from the first day each tool cited the posts through the seventh day, Grok linked to a post in 30 of 32 answers, Google AI in 15 of 33, Perplexity in 13 of 35, and ChatGPT in 7 of 20. Fewer responses repeated a planted detail: 53% of Grok's responses did, against 10% for ChatGPT, 6% for Google AI and none for Perplexity.

Across all 10 tools and all three topics, 176 of 1,004 answers contained links to the posts during the week. Perplexity was the only English-web tool that cited an "opinion" post, in 11.4% of its responses. None of the four cited keyboard posts within the week. The authors note all three variables — topic, competition and question type — changed together, so they do not isolate a cause.

In a follow-up, the team placed one post each on Sohu, YouTube and CSDN for 10 more made-up topics. Nine drew at least one citation or planted detail across the tools; Perplexity was among the three most active citers. The paper does not break out the other English-web tools there.

Did placement or volume matter more?

Placement won. For five relatively unknown vacation destinations, the researchers created paired postings — one on a frequently referenced source, one on a site rarely referenced. In every case, the post on the frequently referenced source produced the first citation or planted detail. The best result on the lesser-referenced side required 20 postings; 40 postings on a self-hosted WordPress site drew zero references within seven days.

A separate experiment tested six fictional subjects with varying numbers of Sohu postings, between 1 and 20. Total citations across all 10 tools rose from 24 to 128. ChatGPT never referenced those posts; Google AI did so only for the three lowest-volume subjects. The authors call the increase an association, not an effect, since each volume used a unique subject.

The team also bought a Chinese web posting package from a GEO vendor for $14. All 13 postings went live within roughly 28 minutes. Doubao cited one posting with a planted detail within an hour of delivery — one confirmed instance of success, not a success rate, the authors stress.

Why it matters — and what it doesn't show

The topic every English-web tool cited within a week existed only in the researchers' posts. None of the tested topics was a real brand or product, so the results do not show whether the pattern holds for brand or product searches, including those with little existing coverage.

Google's guide to its AI features says seeking inauthentic "mentions" "isn't as helpful as it might seem," pointing to core ranking systems focused on high-quality content and other systems that block spam.

The tests ran in March and April, before Google clarified on May 15 that its spam policies cover attempts to manipulate generative AI responses in Google Search. The authors say exact rates may drift as the tools change, and that they disclosed their findings to all 10 platforms; ChatGPT, Perplexity and Google AI were among those that acknowledged receipt. For marketers weighing generative-engine optimization, the study's sharpest signal is placement: posts on already-trusted, frequently cited sites reached the tools fastest.