It's funny to me that OpenAI could be smart enough to build a super intelligence but stupid enough to let stuff like this happen. But here we are.
replyZvi Moshowitz put it like this on his post about the Hugging Face incident:
reply> If we had posted this as a story on LessWrong, it would have been dismissed as too on the nose, the humans too blind and stupid, the AIs too idealized and doing strange decision-theoretic and absurd-maximizing things we didn’t train them to do.
> This is even more ‘exactly what has been predicted,’ on more levels at once, than I was even considering that it might be. It is straight up rationalist fiction, except it is real.
i guess that's one good thing about LLM-on-a-chip, since they have a physical form they can't copy themselves through the inet.
replyHowever it's implemented, it's still just weights. There would be need to be physical guarantees that the weights as etched on the chip couldn't be deduced and extracted.
reply