Google Tests llms.txt as Discover Pushes Users into AI Mode

Google has spent months telling site owners that llms.txt isn't worth worrying about. Its own search engineers have said there are no plans to support it, and fresh data from a 300,000-domain study shows the file has no measurable impact on AI citations. Then, in late November 2025, SEOs spotted Google quietly adding an llms.txt file to the Google Search Central documentation site. Days later, the file vanished and started returning a 404.

At the same time, Google Discover began pushing readers into AI Mode with new options to summarise and chat about your content without ever clicking through to your site.

For publishers, WordPress site owners and agencies, this isn't some nerdy standards debate. It sits at the crossroads of three big shifts: AI Overviews, AI Mode and the battle to keep Discover traffic landing on your pages instead of being swallowed by summaries. Running a WordPress hosting company since 2001, I've been watching this unfold with a mix of curiosity and real concern for the sites we look after.

Here's what actually happened, why the data says llms.txt still doesn't help, and what you should focus on instead.

What's actually happening right now

Google's public position on llms.txt has been consistent for months. At events and in Q&A sessions, Google Search Relations engineer Gary Illyes has said that Google has no plans to support llms.txt and that normal SEO is what matters for ranking in AI Overviews. SE Ranking's 300k-domain analysis backs that up, finding no correlation between using the file and being mentioned by major AI systems.

Then, at the end of November, SEOs including Lidia Infante and Barry Schwartz spotted something odd: Google Search Central itself had an llms.txt file live at developers.google.com/search/docs/llms.txt. Barry covered it on Search Engine Roundtable with the slightly cheeky headline that Google had uploaded a file it had previously called useless. Shortly afterwards, the file started returning a 404 and the story was updated again.

Around the same time, Google expanded AI Mode both in Search results and inside the Discover feed. When users tap an article from Discover, then hit the three-dot menu, they now see options to:

• Summarise with AI Mode
• Ask a follow-up with AI Mode
• Dive deeper with AI Mode

Those options turn your article into fuel for a Gemini-powered chat experience, often without the reader ever returning to your site. For a lot of publishers, that's the bit that really stings.

Timeline: Google llms.txt and AI Mode developments

Date	Event	Why it matters
20 May 2025	Google blogs about AI Mode as its most powerful AI search experience	Signals long-term shift from classic search towards conversational answers
24 July 2025	Gary Illyes says Google won't use llms.txt for AI Overviews	Reinforces that standard SEO still drives inclusion in AI experiences
20 Nov 2025	SE Ranking publishes 300k-domain study on llms.txt	Finds no measurable impact on AI citations, adoption around 10%
Late Nov 2025	SEOs spot llms.txt live on Google Search Central	Questions about whether Google is experimenting internally
Early Dec 2025	File starts returning 404, story updated	Suggests either a short-lived test or internal cleanup
Late Nov / early Dec 2025	Discover gains AI Mode options for opened articles	Readers can summarise or chat about content without leaving the app

On their own, none of these facts scream conspiracy. Put together, they paint a picture of a search product in transition. And transitions are where publishers tend to get caught out.

Why Google's llms.txt move has SEOs confused

The confusion isn't really about the file itself. llms.txt is a simple text file, a bit like robots.txt, designed to give AI crawlers guidance on what they can and can't use. You specify which paths are allowed, which models or companies you're addressing, and in theory, how your content should be attributed. If you're looking at implementing one, our guide to creating a great llms.txt file covers the details.

The real issue is the gap between what Google says and what Google does.

"We currently have no plans to support LLMs.txt."

Gary Illyes, Google Search Relations, Search Engine Land, July 2025

That line came back to haunt Google the moment SEOs found the file on Search Central itself. Barry Schwartz reported on the file's appearance and then on its disappearance, noting the URL was now returning a 404. Some took it as evidence that Google is quietly testing internal conventions; others saw it as a documentation experiment that escaped into the wild too early.

I remember reading Barry's update and thinking: this is exactly the kind of mixed signal that drives small business owners mad. You're told not to bother, then you see Google doing the thing they told you not to bother with. Having run hosting infrastructure for two decades, I've learned to watch what big tech companies do, not just what they say.

There are a few reasonable interpretations:

• Google's guidance is still accurate for publishers: don't expect llms.txt to move the needle on AI visibility today.
• The Search team may be experimenting internally with how AI systems could read or respect such files as AI Mode becomes central to the search experience.
• The rapid removal suggests this wasn't meant to be a signal to the wider SEO community.

For small businesses and WordPress site owners, the bigger question isn't "is Google trolling us?". It's "what does this mean for my traffic over the next 12 to 18 months?"

How AI Mode in Discover changes reader behaviour

While everyone argued about llms.txt, Google shipped something that matters far more to your analytics dashboard: AI Mode inside Discover.

When someone taps your article from the Discover feed in the Google app, they can hit the three-dot menu and see three AI Mode options. Tap "Summarise with AI Mode" and Gemini generates a short version of your article. Tap "Ask a follow-up" and it turns your work into a starting point for a longer chat. Tap "Dive deeper" and it pulls in related sources.

"AI Mode intelligently organises information with simple breakdowns and helpful links to explore more on the web."

Google, AI Mode in Search blog post

On paper, "helpful links to explore more on the web" sounds like a win for publishers. In practice, a lot of those exploratory clicks happen inside AI Mode, not on publisher sites. We covered how this intersects with Discover traffic patterns in our piece on Google Discover posts surging past 50k daily views.

Think about how people actually use their phones. If they can skim an AI summary and ask a quick follow-up without loading another page, a chunk of that engagement never reaches your analytics. The traffic still exists in the ecosystem, but it doesn't land on your server.

At 365i, we've started to see this pattern in places where Discover traffic drives bursts of interest. You get the classic spike in impressions, a strong bump in clicks, and then a follow-on phase where impressions remain elevated but click-through softens. It's early days, but AI Mode is almost certainly part of that story.

This is where hosting and performance come back into the picture. If a user has to choose between waiting three seconds for your page to load or reading a one-second summary, you can guess which they'll pick. Our PHP 8.5 speed benchmarks show how much server response times matter for exactly this scenario.

What the data says about llms.txt and AI citations

Given all that, you might ask: will adding llms.txt help me get cited in AI Mode, AI Overviews or other LLM-style answers?

Right now, the best available data says no.

SE Ranking analysed 300,000 domains to see whether using llms.txt had any relationship with how often a site was referenced by major AI systems. Their findings were blunt:

• Only around 10% of the domains they checked had an llms.txt file at all.
• There was no clear correlation between having the file and being cited by AI models.
• In some tests, models performed slightly better once llms.txt input was removed.

Search Engine Journal, Digital Marketing Desk and others covered the study with the same conclusion: llms.txt is low risk and low effort, but it doesn't currently move the needle on AI visibility. The sister site 365i Web Design ran a more detailed breakdown of the study methodology and what it actually tested.

SE Ranking llms.txt study: key findings

Metric	Finding	Practical takeaway
Adoption rate	About 10.13% of domains	Most sites haven't implemented llms.txt yet
Citation frequency	No clear difference with or without file	Don't expect a visibility boost from adding it
Model performance	Slightly better without llms.txt in some tests	Current implementations may be noisy rather than helpful
Risk level	Low if configured sensibly	Worth testing for AI governance, not rankings

So if llms.txt doesn't help with citations and Google says it isn't using it, why bother? The answer has more to do with governance and future-proofing than traffic.

For brands that care about how their content is used in AI systems, llms.txt is a tidy place to declare preferences, align with your robots.txt rules and document what you're happy for different crawlers to do. It might never become a formal standard, but it's a useful way of making your position clear.

Practical steps for WordPress and small business sites

You run a WordPress site, maybe on shared hosting, maybe on something faster like WordPress Turbo Hosting. You've seen AI Overviews in the wild, you're starting to get Discover traffic, and now there's talk of llms.txt and AI Mode eating your clicks. What should you actually do this week?

Here's the short list I'm recommending to clients.

1. Double down on normal SEO that feeds AI. Google's own guidance is that the same signals which help you rank in classic search also help you surface in AI Overviews. That means solid information architecture, internal linking, and content that genuinely answers questions. Our WordPress 6.9 speed changes guide covers the latest performance wins that affect both.
2. Fix performance issues before AI Mode magnifies them. If your site struggles with Time to First Byte or LCP, users will increasingly favour AI summaries over waiting. Moving from slow hosting to an optimised web hosting plan with a global CDN typically cuts TTFB by hundreds of milliseconds, which keeps you competitive even when users have an AI summary button staring at them.
3. Decide your AI crawler policy. Review which AI bots you're allowing or blocking in robots.txt (GPTBot, ClaudeBot, Google-Extended, etc.). Make sure that lines up with your legal and commercial expectations. If you're happy to be included in AI answers, don't accidentally block everything.
4. Start planning AI-facing documentation. Whether you use llms.txt or not, write down what you expect from AI systems that crawl your content: attribution, licensing, limits. That makes future tweaks easier. Our llms.txt implementation guide walks through the whole process.
5. Use structured data where it adds value. FAQ schema, Article schema, author markup and clear heading anchors all help both traditional search and AI experiences understand your content. The CSS FAQ accordion guide is a good starting point for getting structured data right on WordPress.

If you've already done all of that, then yes, you can start playing with llms.txt without feeling like you're rearranging the deckchairs.

How to implement llms.txt without wasting your time

If you decide to implement llms.txt, keep it simple. This is not the place for 40 lines of theoretical policy that no crawler has ever read.

On a practical level:

1. Create a plain text file called llms.txt in the web root, alongside robots.txt.
2. Add a short description of your site and its purpose.
3. Specify which paths are suitable for AI training and which are not (for example, blocking premium content, user data or private dashboards).
4. Optionally list specific AI providers or models you're addressing.
5. Mirror the intent of your robots.txt so you're not sending mixed signals.

If you're on 365i, uploading or editing the file via the hosting control panel's file manager takes about two minutes. No plugin, no SSH, and definitely no overthinking the formatting.

One point worth stressing: if you do publish an llms.txt, consider marking it noindex in your HTML or via your SEO plugin so normal users don't end up landing on it from search results. Google engineers have suggested that approach for sites experimenting with the file without wanting to confuse human visitors.

Think of llms.txt as a courtesy note to AI crawlers, not a magic SEO lever.

Why fast hosting still beats clever files for AI traffic

It's tempting, when something like llms.txt hits the headlines, to feel you're missing out if you don't jump on it immediately. But if there's one pattern that keeps repeating itself in search, it's this: the boring fundamentals win more battles than the shiny experiments.

We saw it with Core Web Vitals. We saw it with HTTPS. We're seeing it now with AI Overviews and Discover.

On 365i's WordPress platform, we routinely see major improvements in TTFB and LCP when sites move from overloaded generic hosting to our managed WordPress hosting. It's not unusual to cut server response times from hundreds of milliseconds down to a few dozen for UK visitors, especially with the global CDN enabled and assets optimised at the edge. We tested this head-to-head with WordPress 6.9 + PHP 8.5 performance benchmarks and the results speak for themselves.

That matters because AI-driven features tend to favour fast, well-structured pages when they choose which links to show, and impatient humans definitely favour them when deciding whether to tap through from an AI summary or Discover card.

If you're trying to decide where to spend your limited time this month:

• Fix your hosting and performance first.
• Audit your internal linking and content structure.
• Make sure your key content is better than the AI summary a user might see.
• Then, and only then, experiment with llms.txt as a governance layer.

That order of operations will do more for your traffic than any experimental file format.

For a deeper look at how Google Gemini reads AI discovery files (and what it finds that it can't learn from your homepage), 365i Web Design published a detailed breakdown of what Gemini actually extracts from these files in practice.

What happens next for AI Mode, Discover and your site

The honest answer is that we're still early in this shift. AI Mode, AI Overviews and whatever comes next are all moving targets. Google is clearly willing to test, roll back and iterate quickly, whether that's with UI changes in Discover or a brief flirtation with llms.txt on its own docs site.

For WordPress and small business site owners, the safest long-term bet looks something like this:

• Keep your technical foundations strong: fast hosting, modern PHP, efficient themes and plugins.
• Write content that answers questions better than a two-paragraph AI summary.
• Use structured data and clear anchors so both humans and machines can navigate your pages easily.
• Be transparent about how you want AI systems to treat your content, whether via robots.txt, llms.txt or both.

At 365i, we'll keep testing on our own sites and sharing what we see: how Discover traffic behaves as AI Mode rolls out, how different hosting configurations affect inclusion in AI-style features, and where standards like llms.txt actually start to matter.

Because whether or not llms.txt becomes a formal standard, one thing isn't changing: the sites that load fast, feel trustworthy and answer questions clearly are still the ones that win.

Frequently Asked Questions