A randomized field experiment published in August offers the strongest evidence yet that Google's AI search features reduce traffic to publisher websites and deliver a worse user experience — directly contradicting Google's public claims that users love AI Mode and that it sends more traffic to the open web than ever.

The draft paper, titled AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence, comes from researchers at the University of Pennsylvania and Northeastern University in Massachusetts. Unlike most prior commentary on AI search, the findings rest on an actual randomized field experiment that tracked real Google searches, not surveys or projections.

The core findings

The researchers report five main results:

• Google's AI search features reduce traffic to websites.
• AI Mode provided a poor user experience and made users less satisfied with Google Search.
• Removing AI Overviews increased clicks to websites without any detectable decline in users' search experience.
• AI Mode keeps users inside Google's own search experience longer and sends them to websites less often.
• In some cases, the AI search experience was bad enough that users tried search engines other than Google.

How the experiment worked

The study enrolled 1,444 people and tracked the searches of 1,100 of them over seven days. Of those, 956 completed a post-experiment survey. A browser extension recorded search behavior, browsing history and snapshots of Google results pages, allowing researchers to observe searches, clicks, whether AI Overviews appeared and whether participants clicked through to websites.

Participants were randomized into three search experiences:

1. Forced Regular Google Search — AI Overviews hidden, AI Mode searches redirected to regular search.
2. Natural Google Behavior — Google worked normally, showing AI Overviews whenever Google chose to.
3. Forced AI Mode Search — searches redirected into AI Mode.

The experiment hit technical obstacles. "During the experiment period, 94.7% of searches in the AI Mode group were successfully routed to AI Mode," the paper notes. "In the No AI condition, 90% of AI Overviews were successfully hidden on the first intervention day, but the success rate declined to 0% as the experiment progressed. The cause of this breakdown was a change to AI Overview HTML rolled out by Google during the study period that broke our extension's logic for identifying and hiding AI Overviews."

Overall, 51.1% of AI Overviews were successfully hidden, and the median participant in that condition had 50% of AI Overviews hidden.

Effects on behavior

The click-through data is unambiguous. Participants redirected into AI Mode were substantially less likely to click through to external websites. When AI Overviews were removed, clicks to external sites increased. The researchers conclude that Google's AI search features act as a substitute for visiting websites rather than helping users discover external sources of information.

The researchers had predicted AI Mode would increase overall search activity. The opposite happened: users randomly assigned to AI Mode conducted fewer search sessions. "We do not find evidence that exposure to No AI Search impacted search sessions per day," they wrote — meaning AI Mode measurably reduced search activity, while removing AI features had no effect either way.

Effects on perception

AI Mode generally made users feel worse about Google Search. It reduced trust, decreased users' perception that search was useful or satisfying, and made results feel less personally relevant. Removing Google's AI features produced no detectable change in how users perceived Search.

For news-related searches specifically, the study found no clear difference in user perception between AI Mode and regular Google Search.

The effects were broadly consistent across users, with one exception: heavy Google Search users. AI Mode reduced search activity more strongly among heavy users than lighter ones.

A direct contradiction of Google's claims

Google executives, from Search chief Liz Reid to CEO Sundar Pichai, have consistently said users are delighted with AI Mode, engage with it strongly and prefer it. Pichai has also stated that AI Mode is sending more users to the open web than ever before.

This study's findings run against those statements on both counts: AI Mode trapped users inside Google's experience and produced fewer clicks to external websites, while degrading user satisfaction.

For publishers, the data adds academic weight to what many have reported anecdotally — that Google's AI search sends fewer clicks to the web ecosystem, big players and small ones alike. It also raises a broader question about the accuracy of Google's public positioning on AI Mode.

Source: Search Engine Journal (https://www.searchenginejournal.com/research-shows-google-ai-mode-sends-less-clicks-is-a-poor-user-experience/590221/) / Source: arxiv.org (https://arxiv.org/abs/2608.18352)