undefined

points

[-]

llms are not strictly deterministic in the sense that even if you had the RNG state, context, and prompt you would likely not get an identical output even if there was no other randomness involved, because the concurrent scheduling of the massive amounts of floating point calculations can produce different results, since floating point arithmetic is not truly associative [(a+b)+c can differ from a+(b+c)] and the order in which these operations happen can result in subtly different final tensors. To reproduce it deterministically you'd have to also reproduce the exact scheduling of all matrix calculations among all the GPU cores (across different physical gpus!) that it took place on, which afaik is currently impossible.

by Bjartr14 hours ago|

parent|

[-]

That's not inherent, that's a consequence of performance optimizations. It's absolutely a choice to run those matrix calculations in a way that fails to have predictable execution ordering. It's just that the speed benefits to allowing that are considerable.

You can make it trivially deterministic by running single threaded on a cpu, but it's becomes too slow for practical applications if you do that.

by efficax14 hours ago|

parent|

[-]

well sure, but i mean realistically speaking, we cannot step debug an llm's output to find out what happened given the way we currently execute inference

by embedding-shape12 hours ago|

parent|

[-]

Depends on who "we" are, what you're talking about is a thing for inference providers doing batched inference and similar stuff. If you run one inference requests locally, you can actually step-by-step debug LLM output, just there is a ton of steps. But there is nothing "inherently random" or non-deterministic involved here, just optimization strategies for the large inference servers that makes it "impossible".

by solenoid09376 hours ago|

parent|

prev|

[-]

> we cannot step debug an llm's output to find out what happened

We absolutely can with mechanistic interpretability & companies like Anthropic, OpenAI, Meta, and Google do precisely this do debug their models.

by bonoboTP9 hours ago|

parent|

prev|

[-]

It's very possible but somewhat slower. PyTorch and CUDA have flags for determinism. It won't work across all different GPU models though, but it will get you bitwise equal results on the same GPU.

by prohobo8 hours ago|

parent|

[-]

Both of your comments are illuminating :p

So, we could technically debug a prompt's output? I get that there are too many steps to actually step thru, but what if there were checkpoints? At least you could isolate behaviors to specific sections of a neural network?

by bonoboTP8 hours ago|

parent|

[-]

Of course. And mechanistic interpretability research is a thing.

by mmilunic13 hours ago|

parent|

prev|

[-]

Interesting paper by Thinking Machines where they solve this issue.

https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

TLDR: It’s actually more about kernels changing with batch sizes, and you can solve it by making these kernels not depend on batch sizes. It took their inference time from 26s to 42s.

by paytonjjones12 hours ago|

parent|

prev|

[-]

That's very interesting, I wonder if this applies also to models quantized to ints like (-1,0,1), and I wonder if the labs could maintain frontier performance if they removed floating points but arbitrarily scaled up the parameters.

Edit: the Thinking Machines article in the other comment gets into this a bit

by itopaloglu8315 hours ago|

prev|

[-]

We also have engineer blindness, so having human in the loop confirming thousands of requests would quickly start to confirm everything without looking.

It would become just another system to hack through, and slow the development process as well. The OpenAI video in the article recommends an autonomous defense mechanism. For rapid reaction, but I don’t know how sustainable or effective that would be, or if as humans we will be able to keep up.

by layer812 hours ago|

prev|

[-]

> step by step

That’s basically what “pari passu” means.

by andai15 hours ago|

prev|

[-]

I'm not sure I understand. Are you going to debug the neurons?

They are trying to do that, but there are too many of them, so they're building new AIs to help them do that...