What happens between you and an AI chatbot doesn't always stay between you and a chatbot. Over the weekend, users discovered that some **Anthropic Claude** chats were easily retrievable via web search. The problem, initially flagged by a **Reddit** user, exposed a range of conversations, from political advice to sensitive personal inquiries and even erotic role-play. **Claude** offers a feature allowing users to create public URLs, or "snapshots," of their chat threads to share with others. The indexing of these URLs by search engines boils down to a fundamental clash between website functionality, search engine protocols, and the emerging complexities of generative AI. ### The Robots.txt Misstep **Anthropic**'s primary defense against indexing was a `robots.txt` file, a long-standing standard for instructing web crawlers (like those used by **Google** and **Bing**) on which parts of a site to avoid. According to a **Wayback Machine** snapshot, **Anthropic**'s `robots.txt` had designated "shared" chats as off-limits to web scrapers since at least September 2025. However, preventing pages from appearing in search results is more complex than simply relying on `robots.txt`. **Bing**, for instance, still showed "about 612 results" for the query `site:claude.ai/share` at the time of this report. **Bing**'s technical documentation advises developers to not only block crawlers via `robots.txt` but also to include a "noindex" tag on individual pages. Similarly, **Google**'s developer guide states that it will disregard `robots.txt` instructions if a page is linked from elsewhere and lacks a specific "noindex" HTML tag or an `x-robots-tag` in the page's response header. ### The Missing "Noindex" Tag **WIRED**'s review of the exposed **Claude** chat pages confirmed the absence of the crucial "noindex" tag, which both **Bing** and **Google** consider when determining indexing. While some of the initially flagged chats were deleted and no longer appeared in **Google** searches for the specific query, the underlying vulnerability remained. An **Anthropic** spokesperson stated, "We give people control over sharing their **Claude** conversations publicly, and in keeping with our privacy principles, we do not share chat directories or sitemaps with search engines like **Google**. These shareable links are not guessable or discoverable unless people choose to share them themselves. When someone shares a conversation, they are making that content publicly accessible, and like other public web content, it may be archived by third-party services." **Microsoft**, which owns **Bing**, did not provide a comment. **Google** spokesperson **Ned Adriance** asserted that the responsibility for indexing lies with **Anthropic**, stating, "Neither **Google** nor any other search engine controls what pages are made public on the web, and these pages were indexed across many search engines. We give site owners clear controls to decide whether pages can be crawled or indexed, and we always respect those directives." **Anthropic** did not address inquiries regarding the omission of the "noindex" tag. ### A Recurring Issue and Broader Implications This isn't the first time **Anthropic** has faced scrutiny over this issue. In September of the previous year, **Forbes** reported similar findings, highlighting that `robots.txt` alone offers no guarantee against search engine indexing. Despite its limitations for preventing indexing, `robots.txt` remains a common tool for AI labs to manage other aspects of web crawling. Many labs leverage it to signal to creators that their websites won't be used for AI training if specific crawlers are disallowed. Conversely, AI companies like **Anthropic**, **Meta**, and **OpenAI** use `robots.txt` to block their competitors' web crawlers from accessing parts of their chatbot platforms. While shared **Claude** chats are currently not appearing in **Google** results, the fact that the pages still lack a "noindex" tag means they could potentially reappear in search engines. Users of **Claude** are advised to manage their shared chats by navigating to **Settings > Privacy > Shared chats** to ensure their conversations remain private. *Additional reporting by Andrew Couts.*