Harnesses (and the concept of agents before them) presuppose competence in LLMs which simply doesn’t exist.
0. https://www.businessinsider.com/sam-altman-ai-utility-electr...
His idea of metering is predicated on the thing he’s selling being AGI, it is not, and all his predictions have turned to dust.
Also that isn’t how metaphors work - they illuminate by comparison, if the comparison is not close they are not useful.
I don’t believe in AGI, but that doesn’t mean I don’t find AI useful. I just understand that the correct harness can take them to the next level.
Then how do you explain the wild success at using them for development?
That doesn’t make them intelligent agents which think independently.
I have a system that entirely reverse engineers old arcade games. Creates semantic symbol mappings that were considered impossible just a couple years ago.
Granted, it took me a couple weeks to build the system.
From impossible to a couple weeks in just a couple years.
Would you like to see it or continue to pretend these things don't exist? Your call.
(It's finding the coolest stuff - the anti-tampering hacks they put into the old machines is fascinating.)