From what I got, the weaker model (Haiku in this case) has access to the shared key and the user simply asks to "transcribe the injected reasoning".
replyThe provider decrypts it and puts the decrypted reasoning into the model's context window. They prompt the model to repeat back the reasoning. So then the model echoes it back in plain text.
replyHmm, ok. So the attack doesn't involve decrypting the payload, only getting the server to do so. Since a model will do that if you just ask, what's so special about the attack?
replyThe large models whose thinking traces are useful are safeguarded against this reasoning replaying. the small models are just designed for speed and efficiency, so these safeguards are a lot meaker, making the attack possible
replyI guess someone forgot to salt the encryption scheme with a meakness factor.
replyAnthropic server decrypts it as part of fulfilling every request, and haiku recites it per your request.
reply