# Cloudflare lets sites block AI training, not search

> A new Cloudflare setting gives publishers a cleaner way to say yes to search crawlers and no to AI training.
- Title: Cloudflare lets sites block AI training, not search · Pymnts
- Summary: A new Cloudflare setting gives publishers a cleaner way to say yes to search crawlers and no to AI training. Cloudflare is pushing crawler operators to…
- Keywords: cloudflare, ai training, search crawlers, mixed-use crawlers, website owners, apple, google, microsoft, matthew prince, bryan becker, bot activity, know your agent
- Source: Pymnts — https://pymnts.com/connectedeconomy/2026/cloudflare-helps-website-owners-block-ai-scraping-without-losing-search-visibility
- Author: PYMNTS
- Published: 2026-09-16T01:28:02+00:00
- Read time: 1 min
- Topics: cloudflare, ai training, search crawlers, mixed-use crawlers, website owners, apple, google, microsoft, matthew prince, bryan becker, bot activity, know your agent
## A cleaner crawler choice
Cloudflare launched Disallow AI Training, a setting that lets website owners stay visible in search while blocking crawlers from using their content to train AI systems.
## The old tradeoff
Mixed-use crawlers created a bind. Blocking them from AI training also meant blocking them from search indexes, cutting off discovery along with unwanted reuse.
## Access with conditions
With the new setting, mixed-use crawlers can reach a site only if they respect the owner's refusal to allow AI training on that content.
## Big firms line up
Cloudflare says Apple, Google and Microsoft have either met the setting's criteria or committed to doing so. Other companies already split search crawlers from training crawlers.
## Control becomes the pitch
CEO Matthew Prince framed the move as a way to keep search open while giving publishers and businesses more say over how their work is used.

> preserving the openness that makes search valuable
## Bots raise the stakes
PYMNTS Intelligence found nearly 90% of organizations struggle with bot activity, as AI agents and malicious bots make old defenses harder to trust.
## Key takeaway

Cloudflare is pushing crawler operators to separate search access from AI training so site owners can keep visibility without giving up control.