Heh, Signal Detection Theory strikes again! This problem is as old as detecting whether a radar blip is a WW2 bomber on its way or not.

Sadly, there's no perfect threshold when the signal and noise distributions overlap substantially, just different trade-offs.

(Love CI, btw!)