> ## Content Index
> Fetch the complete content index at: https://www.techloy.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Cloudflare accuses Perpelexity AI of sneaking around to scrape website content
- URL: https://www.techloy.com/cloudflare-accuses-perpelexity-ai-of-sneaking-around-to-scrape-website-content/
- Published: 2025-08-05T16:17:50.000Z
- Updated: 2025-08-05T16:17:49.000Z
- Description: This isn't the first time the AI company has been accused of dipping its hands in the data jar.
- Author: Oyinebiladou Omemu
- Tags: / News, Cloudflare, Perplexity.ai, / Artificial Intelligence

Last month, [Cloudflare](https://www.techloy.com/tag/cloudflare/) posed a question: “Why are AI bots allowed to eat the internet for free?” In answering that question, the company then introduced *Pay per crawl*, which requires AI companies to pay website owners every time they scrape content.

Not more than a month later, Cloudflare [publicly accused](https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/) Perplexity of scraping websites that had not consented to be scraped. According to Cloudflare, Perplexity’s crawler hid behind fake user agents, impersonated [Google Chrome](https://www.techloy.com/tag/google-chrome/), rotated through shadow IP addresses, and accessed content that was explicitly blocked. 

Consequently, Perplexity’s bots were delisted from its trusted list.

![](https://storage.ghost.io/c/c1/a6/c1a6d111-d951-41ad-a392-c1e841210b93/content/images/2025/08/image-18.png)

Image Credit: Screenshot from Cloudflare's blog.

If you’re wondering whether this was some random misunderstanding, it wasn’t. Cloudflare set up test domains, specifically hidden from search engines, and Perplexity’s crawlers still found their way in. The AI company was actively working around restrictions, and we're not talking about a few one-off hits. This was happening across tens of thousands of domains and millions of requests per day.

[Cloudflare wants AI bots to pay you every time they scrape your websiteIt could offer website owners a new income stream.![](https://storage.ghost.io/c/c1/a6/c1a6d111-d951-41ad-a392-c1e841210b93/content/images/icon/techloy-avatar-1-2990.png)TechloyKelechi Edeh![](https://storage.ghost.io/c/c1/a6/c1a6d111-d951-41ad-a392-c1e841210b93/content/images/thumbnail/photo-1452830978618-d6feae7d0ffa-1)](https://www.techloy.com/cloudflare-wants-ai-bots-to-pay-you-everytime-they-scrape-your-website/)

Perplexity, of course, denied everything, claiming the crawler in question wasn’t even theirs. But it’s hard to ignore the growing pile of allegations. This isn’t the first time Perplexity’s been caught with its hand in the data jar. In 2024, Wired accused the company of plagiarism.

All of this plays into a much bigger moment for the internet. As AI-powered search tools like Perplexity grow in popularity, traditional search engines like Google are [slowly losing dominance](https://www.techloy.com/googles-grip-on-search-slipped-below-90-for-the-first-time-in-a-decade/). And while users might love getting AI-generated summaries instantly, the websites that fuels those answers are being hollowed out. No traffic. No credit. No compensation.

Cloudflare holds a significant position, putting away the era of free scraping. With Pay per Crawl, they’re giving websites a way to either block AI bots entirely, charge them for access, or let them in for free. Going forward, every new site using Cloudflare will block AI bots by default, a reversal of the open-access era that trained GPT, [Claude](https://www.techloy.com/tag/claude/), and [Grok](https://www.techloy.com/tag/grok-ai/).

As for Perplexity, the damage may already be done. The company’s bots are now lumped in with untrusted traffic, making it harder to crawl sites undetected. And if Cloudflare’s defenses hold up, Perplexity’s AI search engine may start looking a lot less smart without the data it’s been quietly scraping.

[Perplexity Officially Launches Its AI Browser Called CometBut Perplexity isn’t the only one going for Google’s crown.![](https://storage.ghost.io/c/c1/a6/c1a6d111-d951-41ad-a392-c1e841210b93/content/images/icon/techloy-avatar-1-2993.png)TechloyLouis Eriakha![](https://storage.ghost.io/c/c1/a6/c1a6d111-d951-41ad-a392-c1e841210b93/content/images/thumbnail/Perplexity--Comet-2.webp)](https://www.techloy.com/perplexity-officially-launches-its-ai-browser-called-comet/)