All anti-bot measures are inherently flawed, and if you overdo it, your false positives cause real harm by excluding actual people from using your services.
replyNot as in “1 out of 1000 legit requests get blocked” but “you’re going to block 100% of some person’s requests.”
Because I also use AI, and I actually want the content from the site to show up for me, as well as other AI users.
replyMore over, I actually pay for three different AI subscriptions, and use them a lot. I also think it’s fair that I pay for their services. I just want them to also pay for what they use.
If these were actual legitimate bots using a known user-agent and coming from the same subnet so you could throttle their requests it would be fine.
replyBut I’m seeing requests coming in from random IP addresses from all over the world using randomised user-agents. But somehow these are trying URLs in sequential order. A while ago this would be classed as a DDoS cyber attack.