Skip to main content

Search

Items tagged with: LazyWeb


Anybody know anything about the following User Agent strings?

  • ReplicantReaderBot: “Replicant” isn’t an entirely unique brand name. I hope this is unrelated to the Replicant LLM chatbots. If it is, is it used to train or is it just a client of the chatbots?
  • ArenaBot/1.0 (+<https://arena.im/bot/;> contact@arena.im) (page is a 404; is this used to train LLMs or does an LLM use this as a client to fetch data?)
  • SocialBeeAgent: again, used to train LLMs or a client of an LLM?
  • Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5 Tencent/BrandProtection. Does this obey robots.txt or am I gonna have to add another Nginx rule? I normally block brand-protection bots.

#bot #scraper #LazyWeb