upvote

points

by emilburzo9 hours ago |
comments
upvoteby smallerize5 hours ago|
[-]
I understand that it works most of the time. I just want to be clear about what the stakes are.

"Because I have config backups/know how to unbrick?" This is the important part for me. I'm not going to gamble with more than I can afford to lose.

reply
upvoteby perching_aix5 hours ago|
parent|
[-]
The same recovery paths and risk boundaries apply as always.

If anything, these "random number generators" (which they aren't, do not ever use them for that, obviously) are:

- even more cautious,

- more readily take, stash, test and use backups,

- check for and take recovery paths,

- do dry runs,

- consider interruptions,

- mine for more information,

- smoke-test more,

- and are better equipped to do both deeper and wider analyses when encountering problems,

than the typical person.

reply
upvoteby smallerize2 hours ago|
parent|
[-]
I'm being very literal about the randomness. The language model outputs a distribution over all possible output tokens, and the chosen output is sampled from that distribution. The process intentionally includes output that is not the top-ranked token according to the model.
reply
upvoteby perching_aix1 hours ago|
parent|
next
[-]
Think about that for like two seconds.

Yes, randomness is involved, except you're destroying the snot out of it by biasing it to hell and back with those billion-trillion parameter models in front of the sampler.

Alternatively, check the difference between an arbitrary ChatGPT prompt response, and the output of the following:

  import tiktoken, secrets
  
  enc = tiktoken.get_encoding("o200k_base")
  tokens = tuple(enc._mergeable_ranks.values())
  msg_len = 128 + secrets.randbelow(2048 - 128)
  token_ids = [secrets.choice(tokens) for _ in range(msg_len)]
  print(enc.decode(token_ids))
You'll note that they're not much alike. You may also note that with LLMs you can set the "temperature" parameter to zero, in which case the sampling is no longer randomized. So much so, that given various runtime conditions, passing in the byte exact input will return the byte exact output, every time. Possibly across machines, depending on the inference engine used.

Even if you mean it on a "higher level", imbued into the language emitted, do feel free to ask a model with disabled tool calling to give you some random numbers / words. You'll find that it isn't very random. Certainly wasn't when I last tried this for entertainment.

reply
upvoteby antonvs2 hours ago|
parent|
prev|
[-]
No, you’re not being even slightly literal. If language models were random number generators, their output would be gibberish. That’s literal.
reply