Publishers launch pay-to-scrape licensing standard as AI crawlers ignore blocking attempts
Really Simple Licensing 1.0 gives publishers control over how AI companies access their content, with backing from over 1,500 organizations including major infrastructure providers. The standard arrives as websites increasingly block AI crawlers like GPTBot and ClaudeBot, which have surged 70% since July, while 13% of AI bot requests ignore existing robots.txt directives.