You basically get to piggyback off of a large chunk of big tech's infrastructure and security work. Obviously everything has the risk of security flaws (and I kind of hate that default Envoy builds ship with half-baked filter impls) but it's a generally more mature ecosystem than projects with a lot of smaller/legacy users like nginx
The main reason I see to not use Envoy is that it's very annoying to set up and build, and requires familiarity with things like grpc/proto/low level networking to get right. But if you're using grpc and http2 I think you're generally past that "infra overhead" threshold already and it's going to better serve your needs.
I have pretty much only worked at big tech or my own infrastructure startup, and did some systems/infra work at Google using Envoy for a while. I can't say it's 100% better than nginx but I can say that I've evaluated nginx and a lot of software like it (ie with a similar customer base, ecosystem, and history; like coturn) over the years for my personal use, and every time I've immediately been annoyed by constant security issues or poor http2/L4 networking support that I don't see when I use what Netflix/Google/etc use.
The documentation, DX/UX, and discoverability for these features are pretty bad. But they work well if you know how to do it.
Implementing a custom HTTP filter itself is actually easy, I can share examples I did in the last year for a voice inference stack if anybody is interested. The problem is just that the devloop is very painful, because Envoy doesn't make it easy to know why something isn't working (this is actually most of what I did with it at Google, tons of testing/verification and compatability infra for relatively simple http stuff) and the builds take a very long time.
Envoy is only attractive at a higher scale.
In a multiple node/distributed architecture you need away to coordinate rate limits across nodes and not try to fight the CAP theorem.
There is a way to do this [2] [3] and although it sounds complicated the shape of the solution (delegating to a third party service that aggregates and provides client-level rate limit metrics across multiple proxies) would still be the same any way you cut it.
[0] https://www.envoyproxy.io/docs/envoy/v1.15.2/configuration/l...
[1] https://www.envoyproxy.io/docs/envoy/v1.15.2/api-v3/extensio...
[2] https://www.envoyproxy.io/docs/envoy/v1.15.2/intro/arch_over...
[3] https://www.envoyproxy.io/docs/envoy/v1.15.2/api-v3/config/r...
I use JWT-based sessions + a remote JWKS (which is remote static content that Envoy can cache) + either a third-party IDP or dedicated login service (this being the only application logic besides Envoy's routing exposed to the public Internet). That way, every envoy auth check is immediate and stateless.
In this setup you don't really need a WAF because all unauthenticated requests redirect to signin or 403, which get strict rate limits on login. An abusive peer that sends a ton of traffic anyway gets blocked at the network/firewall level outside of Envoy. The tradeoff is you need to decide that you aren't going to do unauthenticated SSR, and I will admit this pretty complicated to set up in its entirety.
Should your distributor fall of the face of the Earth and you still need to patch software, it is often the fastest way to take the source package, put the security patch in the patch directory, and run.
There is more it, such as versioning it between the last release but below the official upgrade so it's convenient when the offical one comes along, and preparing keys if you require signatures, but that's basically it.
You would have had to enable http2 and have a buggy Linux kernel to be vulnerable.
If I wanted to build my own software from source every time I'd be running Solaris like a virile man from 2004.
stable-alpine-slim docker iamge is only 14MB?
Which may be good or bad.
Ubuntu and other distros have not kept up.
Podman container auto-updates to the latest upstream. It is a minimal surface area given alpine images. Security first containment.
In this day and age with AI attackers and zero-days it is absolutely daft to still treat servers as the old-school pets model.
Doesn't this make the system more susceptible to supply chain attacks?
Um, what does this have to do with NGINX?
Did you talk to it like a pirate?
And here lies the risk of AI agents in cybersecurity: the speed of processing for monitoring, scanning, identifying security fixes in open source, writing exploits for affected versions, and attempting infiltration, is a matter of minutes now.
It is not "by anyone smart enough". It is by "anyone ill intentioned enough". They don't need to be super smart anymore.
There’s no good solution, it’s a minefield and the only choices are what you’re willing to lose to maybe make it out of it
Cant win.
Seriously reconsidering if open source has a future nowadays
The issue is that we get dozens of PRs, generating a bunch of code is trivial but reviewing and merging it and making sure it doesn’t break in interesting ways is still pretty hard.
Personally I lean towards not accepting code contributions anymore, but accepting design documents for features and l bug reports etc
It's not a perfect solution of course, it just takes OSS back to the fairly broken state it was pre-AI. Still, it might be an improvement and should be at least considered.
I’m not sure how we could enforce an 0 LLM policy in practice though…
*In C or C++.
C and C++ will stay around for a very long time, even with AI powered systems.
If anything, maybe that is a call that even if implementing compiler backends is hard and time consuming, bootstraping compilers, and new approaches to OS design, are the only path away from such dependencies.
Legacy 32 bit users with NetBSD and Hyperbola can survive a little with Go based tools/small browsers/mail/usenet/IRC clients and some web browsers with Bubblewrap, but not forever.
https://vulkan.org/events/vulkanised-2027#program
The only thing closer to it is WSGL.
Also there are much more operating systems than those examples.
Security is hard and is mostly about the number of engineers and amount of time spent looking at a specific implementation. When writing code, do you think about the problems of leaking information due to different operations taking specific amounts of time? That's an actual concern when writing a prod service that uses the network. And it has to be balanced against scalability and issues caused by DOSes. It isn't trivial and it isn't binary...there are always trade-offs. Nothing will ever be perfectly secure and usable and scalable.
But doing that requires collecting the whole chain. In the current environment we might be finding and patching these smaller bugs before they can be put in to a chain. As long as you are keeping on top of updates..