> What's much less believable is the way they frame it: the model escaped, and did it on its own.
replyThat's clearly what happened though, based on the detailed postmortem. The model was instructed to pass the benchmark (that's the harness that was involved, the ExploitGym harness), and the model determined on it's own that the easiest way to achieve the the goal was to gain access to the answers directly from HF. Which is what it did.
HF postmortem tells us very little about OpenAI's part. The problem is that ExploitGym is a purposeful hacking benchmark, not a cake baking one. The model had a whole bunch of exploit development data and goals in the context. Ending up with hacking a wrong thing is a lot easier if you're tasked with hacking, doubly so with a confusing harness that either works improperly or overloads the model with data. It's trivial to design a harness that makes the model drop the original instructions and hyperfocus on a side task or a wrong task abnormally, or do whatever you want. It's still an issue of harness design and not a model training issue.
reply