It's been done, the ZSNES and Project64 emulators have both had exploits which allowed a malicious ROM to run arbitrary code on the host. ZSNES is written mostly in assembly so that was kinda asking for trouble though.