Google Confirms Robots.txt Can Hide Age-Verification Popups From Googlebot

By James Calloway

Google's Search Advocate John Mueller has confirmed that site owners can use robots.txt to stop Googlebot from rendering an age-verification popup that human visitors are legally required to see.

The ruling came in response to a Reddit question from a self-described non-technical publisher. The user described a hypothetical site, websiteA.com, that loads an age-verification popup from a separate domain — verificationpopup.com — inserted via an API. Their question: would a disallow rule in verificationpopup.com's robots.txt stop Google from rendering the popup?

"That would generally work," Mueller answered. He also suggested an alternative: "If you're using a specific JS script file to do this, you could also disallow that specific javascript file via robots.txt. (If you only have one JS file that does a bunch of things, I wouldnt' do that."

The short version: Google is fine with selective blocking, as long as the page still works without the blocked resource.

Why the question mattered

The question wasn't paranoid. In the early days of SEO, practitioners routinely blocked Google from downloading JavaScript and CSS files to save crawl budget. Then Google started rendering web pages and required site owners to let Googlebot download everything a user would see. Sites that kept blocking those critical resources temporarily lost their rankings.

So asking whether you can deliberately hide a popup — content visitors actually see — was a reasonable check on where the line sits now. Mueller's answer suggests more flexibility than the strict render-everything era implied.

What are the two blocking options?

Hours after his first reply, Mueller returned to Reddit and laid out the two approaches in detail.

Option one: block at the external domain. If verificationpopup.com disallows all crawling in its own robots.txt, Googlebot won't fetch that content during rendering and can't use it. Mueller noted this is exactly how many ad services operate — Googlebot doesn't crawl ad URLs as separate pages, but it tries to fetch them during rendering, and robots.txt stops that fetch.

Option two: block the JavaScript file yourself. If you load the popup with a dedicated file such as "load-verification-popup.js", you can disallow that file in your own robots.txt, and the script won't load during rendering. "In my experience, this is less common, but it's an option," Mueller said. It's the fallback when you can't influence the external domain's robots.txt.

The one condition, per Mueller: "You need to make sure that your pages still load when those URLs are not fetched, but for something like this, that's usually not a problem."

The takeaway is simple — site owners keep real control over which resources Googlebot uses during rendering, even for content that's part of the visitor experience, and Mueller's answer signals Google won't punish it when the blocking is surgical.