Marketer Warns AI-Only Content Creates a 'Shadow Web' Brands Will Regret

SEO consultant Chris Green has issued a warning to marketers optimizing for AI search: content created specifically for AI agents, separate from the public web, risks recreating the manipulation problems that search engines spent decades policing. His argument, published on Chris Green SEO and syndicated by Search Engine Journal, cuts against the current wave of advice urging brands to build machine-readable content estates.

Why does the incentive question matter?

Green opens with a question he says he keeps returning to: why would an AI search provider have a long-term incentive to trust information a brand deliberately keeps separate from its public-facing website?

"There is a significant difference between making information more accessible to AI and creating an alternative version of your website, or even your brand, that exists primarily to influence what AI systems say about you," he writes.

He draws a direct line to SEO history. Search engines did not develop rules against manipulation because they disliked SEOs, he argues, but because letting websites falsify relevance and authority degraded results. His timeline of Google's responses:

• 2011: Panda targeted low-quality content
• 2012: Penguin tackled link spam
• 2022: the Helpful Content system rewarded content created to help people rather than attract search traffic
• 2024: updated spam policies clarified the position on scaled content abuse, regardless of whether AI or humans created it

Google still explicitly identifies cloaking (serving different content to search engines than to users) as spam. Green concedes cloaking's definition is not simple or easy to detect, but says the parallels to AI-specific content are too close to ignore.

What exactly are brands building for AI?

Green distinguishes two categories of practice. The first re-formats existing information: the llms.txt proposal (a structured file directing language models to useful content), agent-specific markdown pages, structured feeds and dedicated endpoints. Cloudflare's Markdown for Agents exemplifies this — the information originates from the same page, presented in a format easier for agents to process.

The second category creates new information that exists only for AI consumption. That, Green argues, is where trouble starts.

"Making the same information accessible in different formats isn't the problem," he writes. "Making different information available to different audiences is where things become considerably more complicated."

He acknowledges why brands find the shortcut appealing. Getting content onto a website has never been easy: content ownership, CMS limitations, design requirements, development resources, governance, approvals and competing team priorities all slow publication. A text file or a separate repository of AI-specific content bypasses all of it.

Can AI systems actually trust what they retrieve?

Green notes a structural difference from traditional search: Google is not the only significant actor. Drawing on his earlier analysis of ChatGPT's search sources, he writes there is no single, consistent retrieval process underpinning every AI-generated answer. Different systems, search providers and retrieval pipelines all shape what information gets discovered and used.

Some inherit established search infrastructure, including systems that evaluate relevance and reliability. But beyond Google and Bing, other sources behave differently. AI systems do not automatically trust everything they are given — yet there is a gap between having mechanisms to identify reliable sources and having two decades of experience with manipulation attempts.

Green also separates retrieval from trust. In his research on how agents behave, they use the least effort to retrieve, summarize and move on; the right answer being available does not guarantee it will be found or used.

He then lists unanswered questions:

• How should an agent establish whether AI-specific information is accurate, authoritative or consistent with what the brand tells actual customers?
• What happens when the machine-readable version says one thing and the public website says another?
• Who decides which is more trustworthy?
• Why should AI providers keep accepting information deliberately separated from what users can independently verify?

"I don't think we have answers to these questions yet," he writes.

Haven't we been here before?

Green rejects the claim that every AI-specific format is cloaking. Structured data, XML sitemaps, APIs and product feeds have existed for years, and he insists brands should give machines information in different formats. The line he draws is between helping machines understand a business and creating content solely to influence AI output, especially content people cannot readily access.

The incentive to manipulate is obvious, he writes, and AI providers' interests run against it: they need accurate answers and user trust, so they have no reason to encourage a fragmented ecosystem of difficult-to-verify sources. He speculates they may validate information against other sources, apply greater scrutiny to agent-specific content, or limit its influence without independent corroboration — but assumes nothing. Treating today's emerging practices as durable strikes him as "worryingly optimistic or naive."

Is a second content estate the answer?

Generative AI has multiplied demand for clear, specific information, Green argues, because users now ask detailed questions and expect complete answers without navigating a website. If that information is missing, AI systems look elsewhere or answer incorrectly. So brands do need to reconsider what they publish, how it is structured and how systems access it.

But he questions why the solution should be another content estate requiring separate maintenance, governance, a possible second tech stack and new consistency checks. "Most struggle to keep one version of their website accurate and useful… I'm not sure creating a second, less visible one is the obvious solution," he writes.

His prescription: make genuinely useful information available to both people and AI, from a consistent, authoritative source that can be maintained, validated and independently verified. Information useful only to AI creates a governance problem; information useful only to people creates a visibility problem. He has previously argued Google's reluctance to endorse separate AI optimization practices makes sense — search providers have little incentive to legitimize an industry of techniques they would then need to police.

His closer lands on the risk: "I'm not convinced that building a shadow web for AI is progress. It may just be another shortcut we'll spend the next decade trying to undo."