There was nothing ambiguous about the task. It had a clear goal and a clear gate to assess success, a gate that takes less than 2 minutes to assess if run correctly (which it didn't until I forced it to encode it as a program).

Your argument that holding the model's hand and telling it what to do and how to achieve the task would make it perform better is ridiculous. It's equivalent to not only giving it the answer to the problem, but also staying all day watching it while it re-derives the solution is correct, making it not only redundant but a waste of time and money.

I remember after a day it made me write in the handoff a hard lesson it learnt after burning 500 dollars of tokens: warming up a synthetic path other than the one the benchmark takes does not meaningfully decrease cache misses. No shit Sherlock, 500 bucks well spent.

It's just literally much dumber than Fable is, which is my argument. I'm not saying it's not usable, I'm saying the difference is quite obvious.

[dead]