Cloudflare is a blight on the internet, but HN loves it.
Data is basically their business.
The question is, what is the alternative to cloudflare? We've tried (to various degrees) user id, micropayments, device attestation... I'm not hopeful we can actually solve this. Maybe if there was a neutral, "NGO-like" cloudflare of some sort?
How about an internet where we don't need CloudFlare?
One without zillions of AI crawlers and script kiddies with Terabit DDoS capability… (the latter being the harder problem)
Imagine archives with static content and user friendly file selection. Like those old world printed product catalogs mixed with content more like books and magazine. One can still have interactive headless websites/api and checkout pages.(Updated) torrents can share folder and file names but you have to force re-check. Could automate that too.
They can be many TBs large, no need to download the entite thing.
But
The llm thing started to change me in a sense that I no longer really feel obligated to use everybody else's solutions. It may be harder and harder to get onto the 'old web', but my new small project put a website from cellphone and is available for as long as I need it to be available.
The short answer to what we can do is 'adjust accordingly'.
There is now a considerable overlap between the capabilities of the dumbest users and the smartest AIs.
There is popular sentiment on HN that CF is making internet worse by being a core infrastructure element most websites use.