By James Calloway

Ask Claude the same airline question in Persian and in Russian, and you get two different airlines. In Persian, Claude picked British Airways 52.5% of the time and easyJet only 12.5%. In Russian, easyJet won 72.5% of responses — a 60-point swing — and British Airways got zero recommendations.

Same model, same scenario, same afternoon. The only variable was the language of the query.

That finding comes from an experiment published by Search Engine Journal on September 7, 2026, which tested whether AI chatbot brand recommendations change with the language the user speaks. The short answer: yes, sometimes dramatically.

How big was the test?

The researcher ran six shopping and booking scenarios through Claude Sonnet 5 with web search switched off, using DataForSEO's LLM Responses API. The scenarios covered a Tenerife hotel, a London–Tenerife return flight, noise-canceling headphones under £400, a robot vacuum under £500, a CRM for a small U.K. business, and project management software for a 50-person U.K. firm.

Each scenario ran in five languages: English, Arabic, Russian, Simplified Chinese, and Persian. With 40 runs per combination, the core dataset reached 1,200 responses. Additional tests with web search enabled, plus runs on ChatGPT (gpt-5.6-sol) and Gemini (gemini-3.5-flash), pushed the total to 1,650 responses.

The recommendation-parsing method resolved cleanly for 99.8% of responses, and the researcher manually checked a sample of roughly 40 before trusting the data.

Which categories swung hardest?

Travel showed the biggest language gaps:

• Airlines: easyJet won 72.5% in Russian but 12.5% in Persian; British Airways never appeared in Russian results.
• Hotels: Bahía del Duque won 43.6% of Chinese responses but only 7.7% of Arabic ones. Hotel Botánico took nearly a third of English recommendations and vanished entirely in Russian.

The other four categories were more stable. The same brand won in all five languages — but the margins still moved:

• Headphones: Sony won 100% of Russian and Persian responses, but only 50% in English, where Bose took 42.5%.
• CRM: Zoho CRM won every language, peaking at 95% in Persian.
• Robot vacuums: Roborock dominated at 87.5%–97.5% across the board.
• PM software: Monday.com beat Asana in all five languages, 51% to 38.5% overall.

Why does this happen?

The experiment builds on Anthropic's July report on Claude's values across languages. Anthropic researchers sampled just under 310,000 real conversations across the 20 most common languages and found Claude's values shifted by language — leaning toward greater warmth in Hindi and Arabic, and greater rigor in English and Russian.

The Search Engine Journal tests add a marketing angle Anthropic didn't test: whether language affects whether a brand gets mentioned or recommended at all. The evidence says it can.

One possible clue sits in the airline data. Russia banned all U.K.-registered aircraft from its airspace in February 2022, and British Airways suspended its Moscow route the same day. Yet easyJet and Jet2 — both British airlines — accounted for 97.5% of Russian recommendations, so geopolitics alone doesn't explain the pattern.

Can marketers do anything about it?

Turning web search on changed the picture considerably. Of 10 language-and-category combinations retested with live search, the winning brand changed in seven.

In hotels, the winner changed in all five languages. Ten new hotel names appeared, including the overall winner, JOIA by Iberostar, which took 31% of picks overall and 60% in English. Iberostar rebranded its Grand tier hotels to JOIA in July 2024; working from memory, Claude had used the old name. With web search off, Claude recommended an Iberostar Grand hotel just twice in 198 responses. With search on, JOIA by Iberostar became the most recommended hotel in the category.

One caveat: 42.7% of web-search responses hit the output limit before finishing — 83% in Persian and 63% in Arabic — so some numbers may be soft.

The tests also found the effect isn't Claude-specific. Gemini matched Claude's winning brand in five of 10 combinations; ChatGPT matched in only one. In PM software, Monday.com won all five languages on Claude but none on ChatGPT, where Wrike took four of five.

What's the takeaway?

The researcher is explicit about the limits: 1,650 responses gathered in a single afternoon across three models, six categories, and five languages is indicative, not definitive. Many hotel-level numbers may sit within the margin of error, and there's no evidence the patterns hold week to week.

The practical advice is direct: run your own tests. Take the questions customers ask before buying in your category, test them across multiple AI models in each target-market language, and repeat the process to see which patterns hold. With live web search shaping outcomes, the researcher concludes marketing teams may be able to influence future responses rather than passively accept a language gap.