Admin Jerry<p>I turned this on tonight. Time to get some popcorn. I hope I'll be able to identify this happening in the logs.</p><p>Trapping misbehaving bots in an AI Labyrinth: <a href="https://blog.cloudflare.com/ai-labyrinth/" rel="nofollow noopener" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">blog.cloudflare.com/ai-labyrin</span><span class="invisible">th/</span></a></p><p>We've seen an explosion of new crawlers used by AI companies to scrape data for model training. AI Crawlers generate more than 50 billion requests to the <a href="https://hear-me.social/tags/Cloudflare" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>Cloudflare</span></a> network every day, or just under 1% of all web requests we see. While Cloudflare has several tools for identifying and blocking unauthorized AI crawling, we have found that blocking malicious bots can alert the attacker that you are on to them, leading to a shift in approach, and a never-ending arms race. So, we wanted to create a new way to thwart these unwanted bots, without letting them know they’ve been thwarted.</p><p>When we detect unauthorized crawling, rather than blocking the request, we will link to a series of AI-generated pages that are convincing enough to entice a crawler to traverse them. But while real looking, this content is not actually the content of the site we are protecting, so the crawler wastes time and resources. </p><p>As an added benefit, AI Labyrinth also acts as a next-generation honeypot. No real human would go four links deep into a maze of AI-generated nonsense. Any visitor that does is very likely to be a bot, so this gives us a brand-new tool to identify and fingerprint bad bots, which we add to our list of known bad actors.</p><p><a href="https://hear-me.social/tags/AILabyrinth" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AILabyrinth</span></a> <a href="https://hear-me.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a> <a href="https://hear-me.social/tags/AIBots" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIBots</span></a> <a href="https://hear-me.social/tags/AIScanning" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIScanning</span></a> <a href="https://hear-me.social/tags/AIBotProtection" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AIBotProtection</span></a> <a href="https://hear-me.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#<span>AI</span></a></p>