might want to edit it

you wrote

in that it’s a vague polite request with incentive for compliance.

I assume here you intended to write "with no incentive for compliance"

yo, I'm looking into additional things to block in the robots.txt for my website (basing a decent bit of it off of yours, plus any additional stuff I find), and I felt like I'd want to just throw this your way

• github.com/ai-robots-txt/ai.ro…
• darkvisitors.com/agents

I'm personally planning to block everything from the first url, as well as the following from the second url

• all AI related tools
• several of the "Intelligence Gatherers"
• possibly several of the "Scrapers"

I would also like to note, BLEXBot is listed on the second site as an "SEO Crawler" and it indicates that it does not believe it is AI-related nvm, I mis-remembered and thought you had blocked it due to it being AI-related

I'll mention any other resources as I find them.

Agents | Dark Visitors

All known normal and artificially intelligent agents

^{Dark Visitors}

ah, I see
did not look at the things you cite lol

do you have some examples of things that are incorrect?

there are several on that list that would probably be good to block, which you don't block (unsure if they're actually used in practice anymore or if they're just historical), such as

• Claude-Web
• cohere-ai
• anthropic-ai

there is also the aiHitBot one that I mentioned