TL;DR
Amazon will enforce robots.txt directives for Amazonbot starting June 15, 2026, replacing manual control requests with industry-standard crawl management.
Key Points
- Enforcement date: June 15, 2026 for robots.txt compliance
- Replaces previous manual request system with standards-based directives
- Allows page-, directory-, and site-level crawl control
- Real-time preference updates via robots.txt protocol
Why It Matters
Website operators gain standardized, ongoing control over Amazon's crawler without manual intervention—a significant shift from proprietary access management. This aligns Amazon with web standards practices and gives developers the same crawl directives they use for other bots, reducing friction in managing scraper access.
Source: xeiaso.net