upvote
Optimize llama.cpp? Hmm.

Wrt speculative decode, basically zero finetunes keep it. I'm testing with some ridiculous abliterated amalgamation so spec decode has been gone for most of its ancestry.

reply