That’s exactly why I don’t believe LLMs will cure cancer anytime soon, make terrible lawyers, shouldn’t be trusted for medical decisions, etc. software and maths are some really the niches where we have great, battle tested, reliable validation tools. That’s not the case for “softer” domains
replyBut this is exactly what the AI labs should be doing ...
replyLove LLMs gonna keep using them. It feels like your suggested approach is expensive, in terms of tokens. I feel (second time I say this) that when I steer the process I get pretty good results vs my coworkers that let the LLMs run away. I do have data on our token usage, not much in terms of quality of the delivery.
replyI keep thinking about the c compiler implementation that anthropic shared earlier in the year that had all the requirements you mention and arguably wasn’t that great.
Thr thing is that both you and your agent should have a way to verify the solution.
replyOBVIOUSLY, the compiler experiment was just a cringe pr stunt. But it has a point: everything works better with a good testing loop, and compilers always have one by thr nature of the work they do
> expensive, in terms of tokens.
replyNo amount of tokens can come close to my hourly rate.
Do you steer your agents by manually running every single test and linter and reporting the results back to them?
reply