Yeah that has been around for many years. Usually by sketchy download sites.

It's not really going to help though because the scrapers using residential proxies aren't burning their own compute.

the brutal truth is that if the website operator simply disabled Anubis, your page load would likely take more than 30s.

when a system was designed for 100 req/s and bots hit it with 5000 req/s, nobody entering that queue is having a good time. Anubis is the trade those operators make just to ensure your request gets serviced at all.

30s load time is already a sign that the Anubis approach is breaking down. if there's nothing else ready by the next order-of-magnitude increase in crawler load, those sites quite likely will just disappear from the public internet. hate Anubis all you want: for most of us, the realistic alternative is strictly worse.

Just look at another tab while you're waiting if you're that bothered.

Visions of vast data centers surrounded by fields of browning grass, in which aging, rusting, formerly extremely expensive hardware is spending billions of compute cycles looking at anime catgirls

It's very likely the last few years of bot behavior is the consequence of the residential proxy business booming. This is indirectly due to AI company crawling, but the fact that they are as cheap and available as they are changes the incentives for anyone using them toward reckless and unsustainable request behavior, as there is no risk of burning your IPs, and very small chances of seeing any consequences of essentially DDoS:ing a website.

Unfortunately we're apt to run into some kind of Jeavons Paradox where the hardware gets so much faster in those few years will be able to slurp massive amounts of data cheaply so the problem never really ends.