Irrelevant to the argument.
> What we do know however is that the open models are downloadable: it's absolutely conceivable that an agent writes a perfectly normal computer virus to gain control of compute worldwide, and uses that control to host instances of its own weights.
These models are hundreds of gigabytes in size, if not terabytes. They don't run on anything close to a regular computer. There's zero risk of self-replication until we live in a world where these "AGI" models are either hundreds of times smaller, or the average computer is thousands of times larger.
Nobody with a datacenter full of H100s is going to fail to notice a parasitic instance of Astra taking over the cluster.
> Unfortunately, there are also multiple AI companies now announcing they've got AI controlling bio labs, so an LLM messing around and making a biological virus is also something we need to worry about.
No, it isn't. This isn't even close to technologically feasible. But the simple answer is simple: don't do that.
If these labs were truly so concerned about this risk, they wouldn't be doing what they're doing.