upvote
The harness is almost irrelevant (you can prove that by running the same tasks in a minimalist harness with only a few tools and minimal prompt vs. a complex harness and long prompt like Claude Code or Codex). Often the minimal harness/prompt actually performs tasks better, faster, cheaper than the big harness. This includes on cyber security tasks.

But, RL improvements are certainly a big part of recent improvements for certain tasks. That's what got the models doing long-running coding and math and research tasks, without constant human prodding.

reply
Completely agree. None of these models are even close to experts in any field (without human prodding).
reply