> how do you properly measure it? > If a problem never manifests, but was prevented, was it really a problem?

For the scheduled half of it, i like inverting the alert. Instead of alerting when a job fails or wait for data leak, have the job check in on every successful run and alert when a check-in doesn't arrive. Dead man's switch. Ten minutes to set up, and it's the difference between "the cron is fine" and a statement that can actually turn out to be false.

Agree that it is difficult to implement for any process, but it is easier to create validation systems than "find the error" processes.