This doesn't answer your question, but I faced an issue where an application had to retry if things failed and I ended up with the "thundering herd" problem and introduced jitter without knowing it was a relatively standard practice.

I felt dirty implementing such a solution (introducing randomness for the sake of randomness is off-putting), but it worked. It wasn't until a while later that I even heard the term "jitter" in this context and realized this was a pretty decent solution for this kind of problem.

Regardless, if you're going to introduce something where adding jitter is appropriate, I'd just just add jitter. It's not premature optimization; it's an essential part of that kind of functionality.

It’s not hard to get started, it’s a case of adding small amounts of randomness.

If you have, say, a long poll then kick off all users due to a deploy or error (or a broadcast message) then you can have a situation where you’ve got a huge clustering of connections at 1 minute, which spreads very slowly out as real life issues give you jitter for free. You can avoid this or at least return to normal much quicker by adding some jitter.

It might happen if all your users back off at the same rate too, if the clustering causes a bunch of errors. Error -> lots reconnect 1 minute after -> fail -> lots reconnect 2, 4…

More likely to occur in cases where there’s a way you can have people all connecting at the same time - synchronisation to a real world event is one case and then connecting again at the same time after.

Have experienced it, but didn’t document.

Downstream database of our edge serverless platform went down. A tonne of requests failed all at once. Every service in the microservice request path, and the client, had their own retry policy.

Clients all retried at the same time. Retries amplified in our microservice graph (1 request at the front door ended up with like 10s of retries internally as each downstream microservice along the path retried requests). Request queues backed up and couldn’t drain fast enough. Clients all timed out at roughly the same time. All waited the same time. All retried again at the same time.

It was a pulsing thundering herd of many hundreds of thousands of requests at the front door that was amplified by internal retries.

Had to tune up load shedding to 100% after the database outage was mitigated until the backend recovered then tune it down in increments to restore service.

Added jitter to clients and turned off retries on the serverless platform.

If you’re talking about internet clients, I think the real world provides sufficient jitter. If you’re talking about a fleet of clients on your 10gbps network, jitter might be useful.

I've had to simulate jitter recently to reproduce a memory spike in a go app using traffic control (tc). have you observed how your app works under jittery network?