Oh I don't disagree with that. I am getting pretty tired of people making multi-thousand-line pull requests with lots of clearly AI-generated code and expecting it to be merged in.
replyI think the better unit to commit and work with is the prompt itself, and I think that the prompt is the thing that should be PR'd at this point, because ultimately the spec is what's important.
> I think that the prompt is the thing that should be PR'd at this point, because ultimately the spec is what's important.
replyThe fundamental problem there is the code generation step is non-deterministic. You might make a two sentence change to the prompt to fix a bug and the generation introduces two more. Generate again and everything is fine. Way too much uncertainty to have confidence in that approach.
If you make the prompts specific enough and provide tests that it has to run before it passes, then it should be fairly close to deterministic.
replyAlso, people aren't actually reading through most of the code that is generated or merged, so if there's a fear of deploying buggy code generated by AI, then I assure you that's already happening. A lot.
How is "fairly close to deterministic" anywhere near good enough? LLMS aren't anywhere near cheap enough to do this either.
replyThat said it's so trivial to do, why haven't you done that already?