# Google bows to pressure, lets websites block AI scraping 2026-08-10T05:52:57+00:00 | Technology | Indian Opinion Desk Corroboration: 1 independent outlet Google will roll out a global setting letting websites opt out of its AI answers without hurting their search rankings, a spokesperson confirmed, following pressure from the UK's Competition and Markets Authority. The test begins after the UK ordered Google in June to give website owners a clear choice to block content from AI products while staying in search results. Cloudflare, which handles traffic for a fifth of the web, will block mixed-purpose crawlers by default for ad-supported customers from September 15. CEO Matthew Prince said Google's crawler sees 3.2 times more of the web than OpenAI's and 4.8 times more than Microsoft's. Cloudflare says AI agents generated over 57% of web traffic versus 42% for humans in 2026, the first time machines surpassed humans. ## Coverage - economictimes.indiatimes.com - hindustantimes.com Tags: AI, Cloudflare, Gemini, Google, UK antitrust, web crawling Canonical: https://indianopinion.org/google-crawler-gives-ai-gemini-unfair-edge-faces-uk-and-cloudflare-pushback/ License: Summary and commentary (c) Indian Opinion, reusable with attribution. Facts belong to the linked sources. Cite: https://indianopinion.org/google-crawler-gives-ai-gemini-unfair-edge-faces-uk-and-cloudflare-pushback/#story-in-brief Review state: restored archive article, accurate at time of publication, not offered to search indexes.