Think about that for like two seconds.

Yes, randomness is involved, except you're destroying the snot out of it by biasing it to hell and back with those billion-trillion parameter models in front of the sampler.

Alternatively, check the difference between an arbitrary ChatGPT prompt response, and the output of the following:

  import tiktoken, secrets
  
  enc = tiktoken.get_encoding("o200k_base")
  tokens = tuple(enc._mergeable_ranks.values())
  msg_len = 128 + secrets.randbelow(2048 - 128)
  token_ids = [secrets.choice(tokens) for _ in range(msg_len)]
  print(enc.decode(token_ids))

You'll note that they're not much alike. You may also note that with LLMs you can set the "temperature" parameter to zero, in which case the sampling is no longer randomized. So much so, that given various runtime conditions, passing in the byte exact input will return the byte exact output, every time. Possibly across machines, depending on the inference engine used.

Even if you mean it on a "higher level", imbued into the language emitted, do feel free to ask a model with disabled tool calling to give you some random numbers / words. You'll find that it isn't very random. Certainly wasn't when I last tried this for entertainment.

No, you’re not being even slightly literal. If language models were random number generators, their output would be gibberish. That’s literal.