I've reimplemented ghostty in Rust and it performed on par with ghostty.

I've also given various models access to the tools necessary to measure performance and instructions to make things faster and they do pretty well. Hill climbing on a validatable measure is reasonably easy these days.

My direct experience here directly contradicts pretty much everything you've said can't be done. I.e. the human out of the loop stuff.