Optimizing for the metric involves:

1. Optimizing for generally applicable skills that the metric is trying to measure.

2. Optimizing adversarially to hill-climb the metric.

You want candidates to do (1) and not (2). You can make them agnostic to the second by setting

    d(expected gain)/d(opportunity cost) = 0
      ==>
    expected gain \propto opportunity cost

It is the case that most metrics are logarithmic: it takes just as much effort to decrease one bit of error as the next bit. So

    log(score) \propto (opportunity cost) \propto expected gain

Thus, for them to be agnostic, you should filter candidates proportional to their log-score on the metric (where 0 is a perfect score). Because generally applicable skills are generally applicable, they will still benefit from improving those, they just no longer benefit from adversarial optimization, unless your score function looks very similar to others who have not adopted this filtering process.

The issue with a hard cutoff is that people near the boundary are extremely incentivized to adversarially optimize, as it is usually cheaper than working on generally applicable skills and actually pays off for them. You see this phenomenon on AoPS where (esp. Californian) students talk about grinding for MATHCOUNTS instead of learning calculus.