For years, AI companies treated the open web like an all-you-can-eat buffet. Crawlers from OpenAI, Google, Anthropic, and ...