No one is actually one-shotting new code on top of the existing repo and just checking it in. Even in a simplistic case, it’s multiturn. You give it a prompt, and it writes some code, tries to build it, and it doesn’t work. So then it keeps iterating, and changes some more code until it builds and then does the same thing again until it passes tests. And if you have a complex system, it might do a lot more CI.

The input to this is not just the preexisting code, the prompt you provide, and some random seeds. It’s also the output from every single tool invoked, which means you need to record the state of every single tool and every file that was read, every API that was called. It means that you can’t upgrade Claude. It means that you can’t upgrade the compiler. If you change anything in the state of the world you no longer have replicability.

Aside from all that, I don’t know why you would want this anyway. Storing a bunch of prompts doesn’t seem to actually gain anything even if it worked.

Except they don’t right? They aren’t deterministic. All the hypemaxxers over at twitter keep saying you need a way for the agent to verify the result itself. Would that be another prompt then?