upvote
That is the opposite of dumb, you can use the next token likelihood to macgyver together a jev in a pinch exactly that way, you just would also want to take some steps to scramble up the choices to smooth over bias and tokenization artifacts. llms have favorite letters for example. Qwen likes A and D.
reply
Yes, you can even do this in unpatched llama.cpp!

It's thought to work "better" but score calibration is still a problem.

reply
deleted
reply