> guessing which tokens a model might generate
replyA transformer that generates tokens based on pre-training? We could call it a GPT for short.
Can we expect similar issues such as spectre and meltdown that intel experienced with speculative execution.. but, in the form of prompt injection/poisoning?
replyOk, I'll bite: no, considering these are very different domains and you don't get system access by getting the wrong speculative branch for your next text token, you just get a slightly different (but probably still related enough) text.
reply> optimizations applied at different levels of the stack
replyThat's because it's just "guess and check" not some deep universal insight.
Simple optimizations are often the best :)
reply