You may as well silently drop the packets if they don't adapt.
Google DNS are busy but its a hand full of IPs and reasonable. Cloudflare is non-stop never ending traffic. I don't know WTF they are trying to get. I've also seen TCP queries from those same Cloudflare IPs so this is not the result of a DDOS attack on them.
1. Breach router or desktop that hasn't been patched in ages via mass script
2. Establish C&C using cloudflare tunnels
3. Sell these hacked systems as residential proxy endpoints to everybody from state actors to teenagers
4. Some of them will be data scraping services
You are chasing your tail trying to ban by ASN or IP. It may work for another couple of months, at best.
If a legitimate request later comes from that address, it will probably get through.
But not sure how well iptables handles checking every connection against a list of possibly hundreds of thousands of addresses.
We had some SSR that followed up with JavaScript listeners to do interactive updates instead of loading a whole new page. But the team responsible didn’t add nofollow and some bots ignore it anyway. Some significant fraction of our cluster size was answering those requests. But one deployment someone broke the fuck out of those links and it showed up immediately in our 4xx error statistics and we rolled the change back within a week.
Three years later I was still seeing those busted URLs being submitted.
Then the fix for the nofollow got implemented and declared done but the flag never got made permanent. We were pretty peeved when someone learned that in some offhand comment in a conversation between teams.
We were already identifying “good” and “bad” bots at the border and telling them to fuck off, so all of this was coming from our supposed friends. And that’s when I realized that a company owning a search engine and a cloud service provider is a protection racket. Google cloud is selling you servers in order to be talked to by Google bots.
Similar to ad click fraud and scam ads, the cloud provider gets paid, why would they turn down the money?
Blocking cloud providers is strictly for us dirty self hosters who are sick of our logs becoming unreadable and our visitor metrics useless from all this malicious traffic.
That sort of thing was very common long before agent based abuse scanners. Volume would wax and wane but never went away.
Though other scans for vulnerabilities have shot up in frequency now everyone script kiddie and their flee-bitten mutt can ask an AI to do it for them.
Why allow random requests coming from the clouds?
However, they could just use residential proxy botnets.
I believe the future is static resources with dynamic apps being only offered to people who can authenticate. And even then I’d recommend grandfathered accounts (eg telegram users that had an account for at least a year).
Agreed
> I believe the future is static resources with dynamic apps being only offered to people who can authenticate.
I don't know about static resources, but I do think that identity will come into play.
Why allow random requests coming from the clouds?
However, they could just use residential proxy botnets.
Traditionally cybercrimes have been prosecuted by the victim going to their local PD (or the FBI) with the IP address of the offender, and then law enforcement subpoenas the ISP associated with that IP address to figure out the customer associated with that account, and then they have a case for prosecution. It all works as long as cybercriminals are less numerous than law enforcement, fall in the same jurisdiction (or in jurisdictions that have reciprocal extradition treaties), and have a real identity that can be tied to their online activities.
Now, many law enforcement agencies are tied up prosecuting political opponents of their bosses. To the extent that they have bandwidth to investigate and prosecute ordinary crimes, they're going to prioritize the ones that target a.) the government that backs them (of which there are many - you see another headline about the government being hacked every week or so on HN) b.) major corporations that have sophisticated IT systems that can aid prosecution or c.) headlines that will get their boss a PR boost and themselves a promotion. Your personal website does not register among those priorities. You probably can't even prove damages from the intrusion itself. The perpetrator might be a Russian bot that the U.S. doesn't have jurisdiction over, or a Canadian teen that the U.S. previously might've been able to prosecute with reciprocal judicial treaties but now has no leverage over because we've blown up our relationship with Canada. It may be legally a crime, but economically it's not worth prosecution.
Traditionally security researchers had to be very careful because they were doing everything above-board, with their real identity attached to their actions, and they made a very attractive (and PR-worthy) target. Somebody who's port-scanning from Russia through a VPN or residential proxy network is a.) anonymous b.) untraceable c.) out of U.S. jurisdiction and d.) not a headline-worthy target anyway.
A couple years ago I predicted here that a lot of things will break as the U.S. transitions from a high-trust to a low-trust society. This is one of them. Much of our justice system works well when criminals are rare and utterly falls apart when they're ubiquitous.