Why does the deterministic nature matter? The interesting part is having oracle tests, not determinism. If someone is deterministic and wrong you use oracle tests to catch that.
replyPeople keep saying "deterministic" when they mean "probabilistic". For illustration, a bloom filter is deterministic, but it's also probabilistic. LLMs are the same.
replyPerhaps a viable approach might be to vibe code the translation tool itself and observe that for every input it gives the expected output. Then once the translation is done, the translation tool can be discarded.
replyThis would require a robust test suite though.
One of the cases where vibe coding might actually be useful, writing a throwaway tool.
I see this dilemma with LLMs all of the time.
replyShould you use the LLM to do the thing directly, or use the LLM to implement a tool that does the thing?
I tend to reach for the latter, it’s easier to reason about.