> This is why LLM-produced text, websites, images all seem so generic.
replyNo, it is not why. That's not inherent to the LLM architecture at all but appears after RL training. Base models don't have any problems with genericness.
deleted
replyWhat? No. Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.
reply> Have you ever worked with programs that do floating-point math on a GPU? It's not deterministic, definitely across platforms, and often within a single platform, too.
replyFloating point math is 100% deterministic, but different hardware/OS have different but deterministic behavior in some corners. The same code run on the same hardware with the same inputs (including access to timers, peripherals, etc.) will behave the same way, unless you're talking about cosmic rays flipping bits or something.
It's certainly possible to write LLM inference on the GPU in a deterministic way, but it's somewhat nontrivial and trades off against performance, so by default most LLM inference engines aren't deterministic even at zero temperature. The classic post about that is https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
reply