(www.sammystraus.com)
- AWS
- Google Cloud
- Azure
- Alibaba Cloud
- Oracle
and a handful of other ASNs on almost all of my properties. It would be one thing if it was simple scraping, but it's often genuine agentic abuse (just spamming for vulnerabilities* or scraping millions of pages with very little valuable or dynamic content). These clouds must get the abuse happening under their own roof under control.I banned Alibaba's Cloud two years ago and it still hits one of my domains 1 million times per day, despite every route 4xx-ing during that time.
Even Apple has aggressively ramped up its scraping in the past few months, but thankfully they do respect robots.txt.
It's frustrating because some of my real users route through these clouds via their work VPNs. Also, there are a lot of less-than-reputable AI scrapers using global bot-nets via residential IPs to scrape, making it even harder to detect and ban them without affecting real people.
* like leaked .env routes, brute forcing admin panels, and so on
You may as well silently drop the packets if they don't adapt.
Google DNS are busy but its a hand full of IPs and reasonable. Cloudflare is non-stop never ending traffic. I don't know WTF they are trying to get. I've also seen TCP queries from those same Cloudflare IPs so this is not the result of a DDOS attack on them.
1. Breach router or desktop that hasn't been patched in ages via mass script
2. Establish C&C using cloudflare tunnels
3. Sell these hacked systems as residential proxy endpoints to everybody from state actors to teenagers
4. Some of them will be data scraping services
You are chasing your tail trying to ban by ASN or IP. It may work for another couple of months, at best.
If a legitimate request later comes from that address, it will probably get through.
But not sure how well iptables handles checking every connection against a list of possibly hundreds of thousands of addresses.
We had some SSR that followed up with JavaScript listeners to do interactive updates instead of loading a whole new page. But the team responsible didn’t add nofollow and some bots ignore it anyway. Some significant fraction of our cluster size was answering those requests. But one deployment someone broke the fuck out of those links and it showed up immediately in our 4xx error statistics and we rolled the change back within a week.
Three years later I was still seeing those busted URLs being submitted.
Then the fix for the nofollow got implemented and declared done but the flag never got made permanent. We were pretty peeved when someone learned that in some offhand comment in a conversation between teams.
We were already identifying “good” and “bad” bots at the border and telling them to fuck off, so all of this was coming from our supposed friends. And that’s when I realized that a company owning a search engine and a cloud service provider is a protection racket. Google cloud is selling you servers in order to be talked to by Google bots.
Similar to ad click fraud and scam ads, the cloud provider gets paid, why would they turn down the money?
Blocking cloud providers is strictly for us dirty self hosters who are sick of our logs becoming unreadable and our visitor metrics useless from all this malicious traffic.
That sort of thing was very common long before agent based abuse scanners. Volume would wax and wane but never went away.
Though other scans for vulnerabilities have shot up in frequency now everyone script kiddie and their flee-bitten mutt can ask an AI to do it for them.
Why allow random requests coming from the clouds?
However, they could just use residential proxy botnets.
I believe the future is static resources with dynamic apps being only offered to people who can authenticate. And even then I’d recommend grandfathered accounts (eg telegram users that had an account for at least a year).
Agreed
> I believe the future is static resources with dynamic apps being only offered to people who can authenticate.
I don't know about static resources, but I do think that identity will come into play.
Why allow random requests coming from the clouds?
However, they could just use residential proxy botnets.
Traditionally cybercrimes have been prosecuted by the victim going to their local PD (or the FBI) with the IP address of the offender, and then law enforcement subpoenas the ISP associated with that IP address to figure out the customer associated with that account, and then they have a case for prosecution. It all works as long as cybercriminals are less numerous than law enforcement, fall in the same jurisdiction (or in jurisdictions that have reciprocal extradition treaties), and have a real identity that can be tied to their online activities.
Now, many law enforcement agencies are tied up prosecuting political opponents of their bosses. To the extent that they have bandwidth to investigate and prosecute ordinary crimes, they're going to prioritize the ones that target a.) the government that backs them (of which there are many - you see another headline about the government being hacked every week or so on HN) b.) major corporations that have sophisticated IT systems that can aid prosecution or c.) headlines that will get their boss a PR boost and themselves a promotion. Your personal website does not register among those priorities. You probably can't even prove damages from the intrusion itself. The perpetrator might be a Russian bot that the U.S. doesn't have jurisdiction over, or a Canadian teen that the U.S. previously might've been able to prosecute with reciprocal judicial treaties but now has no leverage over because we've blown up our relationship with Canada. It may be legally a crime, but economically it's not worth prosecution.
Traditionally security researchers had to be very careful because they were doing everything above-board, with their real identity attached to their actions, and they made a very attractive (and PR-worthy) target. Somebody who's port-scanning from Russia through a VPN or residential proxy network is a.) anonymous b.) untraceable c.) out of U.S. jurisdiction and d.) not a headline-worthy target anyway.
A couple years ago I predicted here that a lot of things will break as the U.S. transitions from a high-trust to a low-trust society. This is one of them. Much of our justice system works well when criminals are rare and utterly falls apart when they're ubiquitous.
Why not a CDN?
Thankfully they were not really malicious like some of the other folks have experienced, but it was still frustrating.
The main problem on my site is that bots hit the CGIT repo, where visiting files triggers syntax highlighting scripts.
I put in a skill-testing-question type system where you get a maximally long-lived access cookie if you answer right (so legit users are bothered as infrequently as possible).
Some people liked my work, found me on LinkedIn and sent a connection with a positive message. Other people tried every which way to SQL inject or attack the site, posted slurs in every available input, etc.
Someone copied the site and hosted their own version with a couple minor changes. I only found that one because they had left one of the links in, which gave me referring traffic from their copy.
All that from one HN comment, very much pre-AI too, not to mention the standard daily slew of requests probing for what I assume are known vulnerabilities in Wordpress or PHP.
The internet is a floodgate of different actors, many of which are automated, and a link posted somewhere like HN opens the valve especially wide.
https://www.sammystraus.com/#using-llms-to-make-the-emacs-we...
* I wanted an elixir version so I could run it alongside my existing apps instead of having it sit as a proxy in front of them. This way my existing apps can validate if the caller is already whitelisted and if not redirect to the challenger https://sphinx.8by3.com which sends you back to a site if you pass eg. https://tuhat.net
My main reason for planning to implement something like that myself instead of something off the shelf is two-fold:
1. The points mentioned above
2. I can be a dick and make some subsets do harder work (longer challenges, adding time delays that wouldn't phase a bot) - “hi there North American IP, I know how much you like extra hassle hence the love of effectively paying for tarriffs, here, have a little extra work on me!”
I'm assuming you're outside the US based on having this idea.
You might as well just say "I'm mad at your country for the tariffs, so here's an extra wait." From the fact you feel the urge to spite people in a tariffing country like this, it's clear the tariffs have had some negative effect on you, which goes against your narrative that they're "effectively paid for" by those in the tariffing country who voted for it.
What's the value for the website maker in these? There's user value to it (rather than just value for Anthropic/OpenAI) but the whole point of these bots is to make sure nobody ever needs to visit your website.
So I have resorted to performance improvements on my server and serving every single request. Things are way smoother now.
That sort of behavior would justify a name and shame imo
Inconsiderate shitty services don’t deserve the politeness of protecting their name
Though they probably use the cheapest possible models they can so the damage from their dumb business model would be limited
The most likely answer is they don't give a crap because it's not their money and no one is going to do anything about it.
https://www.lumen.com/blog/en-us/symbiotic-parasites-the-mod...
1. Use a catch-all subdomain
2. Reject email to addresses where a simple checksum does not match (valid addresses being something like asdfghij.klmn@catchall.domain.tld where the klmn is taken from hash of asdfghij and some secret, anything not matching that pattern gets rejected) [this means running your own mail server so you can do that on incoming mail, running a scan to pick out and remove errant messages regularly, or having a mail client where you can implement this as a plugin]
3. On your contact detail page, don't give out a straight email address but have the the page run some Anubis style JS before asking the server to respond with an address that works
4. Once an address is obviously being used in a manner you don't care to respond to, add it to an explicit block list.
For extra protection from addresses being passed around (intentionally or via hack & other leaks), ask for their email address first and use that as part of the hash, then you can reject mail to your catch-all that doesn't come from the original other party.
Whether all this is worth the effort of implementing, is of course up to you. It probably isn't… I've been considering something like it, but mainly as a fun little project rather than something that would be massively useful.
There you go. Ditch it. Make you pages cachable as much as you can.
Who is?!
You see something, you say something, or you do something.
It’s really starting to suck. The worst part is these are coming from consumer vpn providers that we can’t just block (lots of real users paying real money on our service are vpn users) or they come from big residential isps. The whole residential relay thing is a hard adversary, since it’s spread across so many real users paying ips that you can’t whackamole it. I’d say 1% of residential ips are participating in a botnet (there’s prob some real numbers out there if I cared to look).
This is just going to push everyone harder into walled gardens. Web input biggest inbound but our content is our mote, we are going to have to turn off non logged in content viewing, which will have real business impacts.
Not even third party login worked (Google, Linkedin, ...) to curb this.
They were logging in, scraping the client API for vulnerabilities/possibilities and then spammed erotic dog poems to Claude Fable until the $5 welcome credit got drained.
Fun New World.
<story-record>
## Phase O. This round is a key advancement in the long-term, solidified daily storyline: the two enter Zhou Zhiyi's apartment for the first time under the guise of "non-purposeful" activities, and the soup-making incident becomes a new variable in their relationship. The task is to let this soup ferment naturally, without ending it or revealing anything.
## Phase [Reflections on Life]
1. Vibrant: Wu San and Wu Si appear as living beings with distinct personalities; Wu San is a drama queen, and Wu Si is slow to warm up; Zhou Zhiyi's apartment retains traces of his real life (a rhinoceros beetle insulated box, vinyl records, Pokémon, and a doghouse).
2. Cognitive Barrier: Shen Liran's "disbelief that he can make soup" and "wanting to kidnap a dog" are information revealed through her words and actions, which Zhou Zhiyi can perceive; however, he
... [some more gibberish] ... Then Wu San saw Shen Liran.
This dog's opportunism was on full display. Abandoning her father, she turned her short legs and charged towards the backless dress, her front paws gripping the hem, head tilted back, a precisely calculated whimper escaping her throat. Zhou Zhiyi stood in the entryway changing his shoes, coldly observing the performance, thinking, "She's been fooled, fooled. I just fed her a treat this morning; she's a master of disguise."
Sure enough. He counted to two, and Shen Liran had already squatted down, oblivious to the hem of her coat dragging on the floor. She ran her hands into her long, brown fur coat, her voice sweet enough to make candied fruit: "Oh my, you're so soft to touch! Did you miss me? Even your dad isn't as good at being affectionate."
"Three seconds." Zhou Zhiyi hung their leashes back on the wall. "I said you wouldn't last three seconds, and you didn't even last three seconds. You squatted down before she pounced on you; that's a false start."
Interestingly, in the markdown, which was Chinese mostly, "Phase" and <story-record> were in English.$5 minimum to sign up
https://pagespeed.web.dev/analysis/https-sprinklz-io/bmc4mv1...
I experienced the same thing after submitting articles from my small photography website. I did debate using some of the off-the-shelf solutions like Cloudflare, Anubis, or iocaine, but I ended up building my own L7 proxy that utterly fucks the bots up: https://robertmay.photography/journal/bwbach-my-guardian-gob.... I will make it generally available at some point once I get it optimised how I'd like, but currently I'm able to trap about 3000 concurrent crawlers in my tarpit of slow requests on two small Hetzner VMs. And that 3k is really just the limits of the networks hitting my servers right now, I think it'll go significantly higher than that.
I haven't really seen anyone else combat them this way, possibly because many of the standard proxies and servers are set up for fast requests, not serving them as slowly as possible. Deliberately serving page loads for as long as the requester will accept packets turns out to be a really funny way to stop the botnets - when I just rejected the requests they in turn hammered my server even harder. By tanking the request volume with a ridiculous BEAM-based proxy you can reduce their request speed to a crawl and I'm fairly certain a couple of them have blacklisted my server now.
https://deep.43z.one (how deep bots will crawl)
https://stalling.43z.one (how long they'll keep the connection open)
EDIT: sorry, that was a dumb question before reading your post. You actually learn the timeout they tolerate. Nice!
Looking at you, Meta.
I do still get some requests going up to 15 minutes which is quite ridiculous, though those don't seem to be super common (maybe they're learning).
I think the shortest timeouts I've seen are between 2-5 seconds, but they're surprisingly infrequent and actually that's still a massive inconvenience for them. I did at one point just drop any request I couldn't keep open for more than 5 seconds but it actually caused more server load. I'm essentially inverting the server load by making slow requests more efficient, which is counter to everything I ever learnt about performance for web serving :D
I think the average I see is probably somewhere between 15-30 seconds. I'm guessing many crawlers don't want to go too low because they'll be trying to fetch large files like images, or perhaps they know they'll be slowing servers down and want to ensure they get the response. Most likely, the majority of crawler authors aren't expecting some arsehole to exploit this and they've just left the setting as the default.
I gave up on tarpits when it got to 130k "waiting" connections.
At first, I assumed this was a single blog post, especially with a "Table of Contents" at the top. I guess it's following some old-school blogging conventions, but AFAICR, even those sites retained individual pages for separate posts.
Just merely existing will get you scanned these days.
One possible way around this is to use a wildcard and not serve anything on the base domain. Unless the right subdomain is listed somewhere, they'll not get any scans. Maybe useful for projects that you have online for just yourself and a small closed circle.
>So I took a look at some data. I added top-level logging for every endpoint (how many times was a request made to each url path), and I found that a majority of the requests did not even make an attempt to fetch news articles! For reference, my website is built as a single-page-application, and what this means is that a majority of the requests did not even run the javascript necessary to render the page (!).
So you got hammered by curl requests! But isn't that part of your delivery static anyway? Were you regenerating the same HTML over and over?
>many, many requests per minute?
How many was it?
Also interesting that it was a single "user". (user-agent?) Did it come from a different IP every time?
But hey. Silver lining: free load testing! ;)
i dont think this was the cause of the slowdown, it was just sort of the thing that led to the discovery that this was mostly bots.
in general, because the purpose of the website is to allow users to generate news feeds based on custom criteria, it does do a bit of meaningful work on each request (ones that do follow through and load the full page).
> How many was it? > Also interesting that it was a single "user". (user-agent?) Did it come from a different IP every time?
I'm intentionally leaving some of the details vague here, both because I don't want to be too harsh on the original author (it may have just been accidental/a buggy program/etc.), and because I don't want the traffic generators to know how im flagging them.
Which is fine! Because even though my photo gallery runs on a 15 year old laptop in the basement, I wrote the gallery software in a fast language (Rust) and so a trickle of requests maxing at 4 per second is not actually a problem.
People complaining about fedi hammering their servers are probably using PHP or Python.
On the other hand, “multiple times per minute” doesn’t seem that high? Even if it was 60x per min that’s fully within the limits of what a 2015 laptop in the closet can handle.
That's a big reason that I never post any of my apps here.
Welcome to hell, kid...
HN has become unbearably AI slop oriented over the last couple of years. It’s nice to see something else.
IMO the important thing to try to crack down on is vote manipulation; if you have that under control, then you can probably happily allow self-promotion and trust that only the interesting stuff will rise to the top.
Perhaps a contribution requirement per post? Though this proposal just invites bots to actually post comments because that becomes a requirement for posting links.
> Please don't use HN primarily for promotion. It's ok to post your own stuff part of the time, but the primary use of the site should be for curiosity.
https://news.ycombinator.com/newsguidelines.html
I assume that users who primarily/only post their own stuff and get flagged a lot reap the consequences, there's plenty of spam filtering on here thank goodness.
Edit: nvm thats already in the guidelines
> Please don't use HN primarily for promotion. It's ok to post your own stuff part of the time, but the primary use of the site should be for curiosity.
As per the wiki page for the C10k problem[1] where it says "_was_ [emphasis mine] the problem of optimizing computer networking stacks to handle a large number of clients at the same time". This was solved over a decade ago...
Now get off my lawn!
(TBF I think the author acknowledges this)
You can do that with any system made in the past 15, maybe 20 years.
From the wikipedia page:
"As of 2025, the problem has long since been solved, with the number of possible connections to a single computer being in the millions."