Skip to main content

Search

Items tagged with: iocaine


ROFLMAO.

Claude decided to crawl one of the sites on my new server, where known bots are redirected to an #iocaine maze. Claude has been in the maze for 13k requests so far, over the course of 30 minutes.

I will need to fine tune the rate limiting, because it didn't hit any rate limits - it scanned using 902 different client IPs. So simply rate limiting by IP doesn't fly. I'll rate limit by (possibly normalized) agent (they all used the same UA).

Over the course of this 30 minutes, it downloaded about ~300 times less data than if I would've let it scrape the real thing, and each request took about the tenth of the time to serve than the real thing would have. So I saved bandwidth, saved processing time, likely saved RAM too, and served garbage to Claude.

Job well done.