The big nasty AI bots use 10s of thousands of IPs distributed all over China
replyMillions and all over the world
replySo... just blacklist all China IPs? I assume China isn't the primary market for most of complaining site-owners.
replyA lot of compromised home devices and cheap servers proxying traffic, from all over the world.
replyIf that is the case how can you determine the reason for the activity?
replySome fake user agent, some tell you who they are. Or.. do they?
replyHere-in is the problem. And if you block them, you risk blocking actual customers.
If they are using appropriated hardware, what possible reason could there be for them saying who they are?
replyThree different "companies" normally:
reply1. The residential proxies
2. Scrapers, on behalf of or as an agent of the data buyer
3. Data buyer (ai training)
Scrapers are buying from residential proxies, giving the data buyer a bit of a shield/deniability.
The scrapers don't want to get outright blocked if they can avoid it, otherwise they have nothing to sell.