upvote
SPRT is used religiously in engine development today. There is enormous incentive to test efficiently.

https://github.com/official-stockfish/fishtest/wiki/Fishtest...

reply
DOE is still very useful in many contexts, but when it's possible do use a sequential design these adaptive techniques really start to pull away in terms of optimization quality.

There's simply a lot of sample efficiency to gain by adapting the experiment to incoming data in a regime where one can repeatedly design n candidates, observe their effects, and repeat m times compared to a setting where one must design a fixed experiment with n*m samples.

reply