upvote
Yes and that is just the bot background radiation of the internet. I run a primary source of information site and these botnets are aggressive to a DDOS level. All to do some sort of scraping. Because they have sophisticated enough tactics to DDOS us if they wanted to. However I am not sure their objective as they have wasted enough of our resources to have scraped all our content 1000s of times over. That 25k traffic is a couple of minutes for us. And that adds up. 80-90pct of our traffic is this
reply
Assuming that the bots aren't repackaging your content and preventing users from seeing your blog by serving that content to them first.
reply
True. But it still wastes your server resources, right? And it's sad that you have to accept that as part of the "cost" of hosting a site ...
reply
What resources are you concerned about? An n100 minipc should be capable of serving something like a blog at 20k+ requests/second (or saturating its network).
reply
deleted
reply