The vendor did get the hint, and the second iteration of the device did slowly warm up the pumps so this process would no longer be required.
https://onccy.com/how-to-choose-between-type-a-b-c-and-d-min...
Notoriously a crescent wrench makes an excellent 1000A slow blow fuse
You can observe this with a speed control knob/ESP32 where a running fan will happily run forever with the knob turned down to e.g. a 30% duty cycle. But after turning to 0% and waiting for the blades to fully stop, if set back to the same 30% they won't move at all or you'll maybe hear a sketchy clicking/whine sound with the blades intermittently twitching.
It's sort of a solved problem - the ones I'm aware of are things like LynTec sequenced panels built exactly for this, and Siemens' SENTRON semiconductor CBs where you can control then curve to your spec.
Convincing stakeholders though, is almost a job of its own.
Secondly, the amplitude drops. Events that were concentrated on the xx:xx:00 times are now distributed into xx:xx:xx times. If the overall number of events is the same, each :xx time slot now receives 1/60th of the events that were concentrated on the :00 time. If power decibels are relevant here (20 log 10 x), that's a 35 dB reduction in amplitude of the driving signal.
35 dB less amplitude, shifted to a 6 octave higher frequency that the grid doesn't react to very well can be expected to basically make the problem vanish.
This is much like the situation of soldiers avoiding rythmic marching when crossing a bridge to randomize their footstrikes.
You want your jitter to be significantly >> than the time your task takes as otherwise you get non intuitive degradation that can reintroduce the problem you’re trying to solve. It’s a balancing act picking the jitter that drops off the thundering herd enough and one that is still greater than the time your work takes.
But yeah, as soon as you start using Rand most people make all sorts of mistakes. And if you’re using C Rand, don’t. It has other statistical problems.
The other thing to watch out for is you’ve still got 1/60 of the thundering herd at the second level. If the tasks are <1s and you have enough of them (as you probably do at grid scale), then doing it finer grained is warranted like offsetting the timer by a millisecond sleep of 10 * uniform_distribution<int> between 0 and 6000. That then gives you 10ms granularity, distributing the workload by two more orders of magnitude.
I should have said, you almost always want to use the randomly staggered timers on initialization, not just when the exact timer interval matters! It's also important to test electricity or backend outages, which can unintentionally synchronize all the timers.
A common aggregation is about 10ms although most OSes can give you finer granularity accuracy if you ask for it and have permissions. It really depends on how long the task takes and whether there’s any non-uniformity in its processing to make finer granularity helpful or harmful.
Unfortunately some people do read advice like it’s mana from the gods instead of thinking through the implication on their specific problem.