So I asked Claude to first figure out a way to instrument GPU particle code so it could itself check his solutions. After that I left the agent running for 3 hours and it came back with a solution (continuous collision detection) and a bug fix (particles get their velocity applied an extra frame after colliding). I'm sure someone would find something to complain about the code (which is why I haven't upstreamed it) but it looks visually perfect and I had plan to fork the engine for my project anyway
Epiq solves this with an architecture that supports workflow auditing, allowing you to time-travel state in a filtered view to reconstruct what happened, when, by who, and where intent started drifting, while also allowing you to correlate the evolution of the board with the corresponding commits.
You will get a solution that works with a proper workflow, but you won't get one that scales or would be truly maintainable. Which is also what you get with random midwit drive-by contributors, but faster. I'll give it that.
But more importantly: isn't what you call "intent" just a series of optimization goals? You want your code to satisfy the constraint of being correct™, while also maximizing various other goals like being maintainable, easy to understand, having few lines of code, as little tight coupling as possible, etc. Goals that often conflict, but when given two implementations you could likely tell which hits the better tradeoff (in your engineering experience)
Those are all things that theoretically - with a tight enough specification and enough compute - a constraint solver could solve. No human intent necessary.
The issue is more that we can't fully specify all those side goals, and even if we could the LLM would struggle following them. A classic paperclip maximizer problem (where nobody told the paperclip maximizer to keep the planet inhabitable and all the other side conditions we implicitly assume)
The good thing is that you don't have to agree with that, as the fundamental technical reality does it for you. LLMs do work like that. They are just statistics and probabilities.
If can put properly engineered intent in the prompt that is verifiable, it works wonders. Anything that can defaults to the llm doing its way, you're right, it just can't converge to good, not with proper constraints.
So I wonder whether, in your experience, the results you've seen, could have improved by providing sufficient context? - or what context was given.
I.e. if you have the agent that same context, as one of your colleagues would have/require to solve a problem.
You've missed my point. I didn't dismiss agents. I did dismiss the industry.
I don't need to add more context to a statement that operates on a layer above where context injection would influence it. It is a conceptual impossibility. Not a technical roadblock.
Holy crap. Does your home have mirrors?