AI robots.txt

Earlier today, I stumped on a Mastodon post from Chris Coyier sharing a link to what appears to be a community-maintained `robots.txt` which blocks known AI-related user agents from ingesting content.

This is by no means a guarantee that that AI bots won't ingest this content without consent anyway. As Chris mentions, it's the Wild West out there - content aggregating websites and Big Tech corps to sort of "appropriate" and then monetize the words, images, photos, sounds, voices, videos, movies, software and really everything they can get their crawlers on.

If you're interested in adding this to your blog or online publication, feel free to check out the ai-robots-txt repository.

And if you want to read more on the subject, the maintainer of ai-robots-txt has this excellent post.