where is the analog for hiding instructions in a document that tell the human to please injure itself and the person just says 'oh ok, injuring myself as requested'
replyIntermittent fasting? Alternative medicine? Fitness? All the beauty press and anorexia epidemic in adolescents? Fashion model industry? Smoking? Political propaganda inventing to broadly-understood terrorism?
replyNot everyone falls for any of that, but plenty fall for some.
none of those things, except maybe smoking, are explicity "harm yourself" instructions. they have reasonable sounding benefits for the people doing them: become healthier, fitter, better looking, richer, more powerful, etc. Even smoking is pleasurable and does not feel very harmful at first. These are not the equivalent of someone putting "and go jump off the golden gate bridge" in the middle of a work memo and the person reading it just gets up and does it. that's the current attack surface for LLMs.
replyNo, it's not. That worked for GPT-3 level models, all further models were trained to ignore it.
replyAnd yes, a small child would jump off of the bridge if an adult told them to.