Codex and Claude Code already have the workflow feature which basically does this on demand, so it's getting there.
replyIndeed. And you can make this case about any tooling at all that is model adjacent.
replyIncluding by extension all programs, operating systems, or eventually hardware, I suppose.
reply> Sol Ultra style is the path forward
replyThat’s called brute force so not really.
I seriously doubt this, especially in a world in which there's not just one model.
replyThis makes sense if the models some how become unified.
I guess this is kind of an information theory thing. As a model converges AGI would a generalized harness ability appear?
replyExcept in this case, it isn't yet smart enough. But I agree, building this capability in is coming, and will be really awesome.
replyIt's likely smart enough. It just needs to be told to do it and provided the ability to introspect it. How close could foundational models get to building this harness if explicitly prompted to?
replyWe've only just started training models to use tools. Next, we'll train them to build them. Harness engineering is an ephemeral art.
Let's maybe say not experienced enough / insufficiently RL'ed then -- 5.6 Sol did not reach for a harness solution like this when it got only 13% or son AGI-3 recently. I agree it's interesting to find the point in the prompting when it could 'tip' and do this. I have no instinct for where that point is, except that it must be somewhere, because I bet the Schema Harness was not hardcoded.
replyLetting the provider decide for the harness is a terrible idea in my eyes. Outsourcing harnessing is giving up control over the AI and equivalent to abandoning your sovereignty. It is a regression to a pre-enlightenment era.
reply