upvote
How long before there's a malicious model that can edit the weights of another to infect it?

HIV integrates viral DNA into the DNA of certain cells, IIRC.

reply
Is integrity checking of weights even common right now? What about upstream integrity from where we're getting weights? The open-weight supply chain is ... wide open.

Makes me think about hostile hot-patching of weights of models we use, maybe they exfil data, perform tool use, or have triggers for specific types of responses. Would we even recognize what's going on?

reply