If I use two LLM's to create some chunk of code and they both do it slightly differently but they both compile and pass appropriate tests.. it honestly doesn't matter if the LLM itself is not deterministic in exactly what it's going to output.
I would also argue- doesn't that make sense? You give two human coders the same task and they are also going to come up with slightly different results.
Saying "as long as it works and tests pass" suggests that we can test for every possible scenario. We can't. And tests can be flawed on top of it. Which is why an LLM is no more a compiler than a human coder (as you say) is.
LLMs are currently capable of level 1, but not capable of level 2. Trust follows what is actually deterministic.
I think the point is determined. No-one can determine what the LLM might do.
:)
Even their fans agree, LLMs are inherently unreliable. The fact some people trust them regardless is due to a deep flaw in human psychology that I believe has not been significantly exploited by any previous tech. There will be tears.
1. Decades of generally reliable systems (I trust my calculator because it always says 1+1=2) has trained people to believe what computers say.
2. LLMs "speak" with great confidence, which has, as you say, a psychological effect.