upvote
Record-and-replay debuggers like rr and UndoDB are designed for exactly this scenario. In fact it's way better than logging; with logging, in practice, you usually don't have the logs you need the first time, so you have to iterate "add logs, rerun 600 times" several times. With rr and UndoDB you just have to reproduce once and then you'll be able to figure it out.
reply
I'm not going to manually execute the bug in a test once if it is 1% (or .1%, which I often have to deal with also). I'm going to run it 600, 1200, or maybe even 1800 times, and then pick bug exhibitors to dissect them. I can imagine that these could all be running under a time travel debugger that just then stops and lets me interact when the bug is found, but that sounds way more complicated than just adding log messages and and picking thru the logs of failures.
reply
Trace points do exist.
reply
conditional breakpoints, watches, …
reply
... will sometimes make the race condition not occur because things are too slow.

Like the bugs "that disappear in a debug build but happen in the production build all the time".

reply