upvote
> It is an access issue

Column A, Column B. Building a small explosive device isn't hard. Building a million is very difficult, doing it covertly virtually impossible without the resources of a nation-state.

The problem with biologics is the self-assembly and replication machinery comes for "free." So the numpties who might otherwise blow up a trash can [1] now have a real chance of taking out a million people.

[1] https://en.wikipedia.org/wiki/2016_New_York_and_New_Jersey_b...

reply
The problem with biologics is that you cannot build a virus in your garage. You need a lab. AI will give your recipe, but you still need a lot of money and cooperation of other people (and if you have so, you could hire human biologist in pre-AI era).

Also AI makes mistakes. If you ever coded with AI agent you know that loop "write trash => compile => fix compilation errors => repeat" (if there are no compilation errors, there are definitely logic errors to be fixed). In real world cost of attempt is huge. You need a lot of money and you risk to draw a lot of attention if you perform long series of iterative experiments to create working virus.

In case with bomb it means that even if you have AI which gives you recipe of the bomb, but you will explode your garage and yourself with a decent chance. So you probably need to setup a good experimental pipeline (hardened lab where you can try different formulas and see that happens without being killed) if you want to go beyond publicly known explosives available in pre-AI era to anyone who read school/university chemistry books. And this also requires resources and draws attention.

People extrapolate programming experience (the area where experiments are cheap, cannot kill you and provide detailed feedback what went wrong) to real life.

reply
They would still have to procure things that would (I hope) light up many screens before they're able to. And such numpties are probably already monitored, or in prison for some other stupid life decision.

I also would like to hope that people that are likely to do such things are probably:

A) don't know how to break even the most basic guardrails of models

B) already in glasswings project

To prove point B - Theranos existed.

reply
> They would still have to procure things that would (I hope) light up many screens before they're able to

“Many of the largest and most responsible providers in the industry already screen and record orders voluntarily,” but there is no requirement to do so [1].

[1] https://screendna.org/

reply
> ...you bet your ass...

Humorously, whether I choose to participate in this hypothetical or not, I am already betting my ass.

This whole situation feels like the game [1].

[1]: https://en.wikipedia.org/wiki/The_Game_(mind_game)

reply
Why. That was just uncalled for. Sigh
reply
If that were possible, they would already be attempting it with the same level of ability as if they didn’t have access to a text file generator app. It is not about access to the information.

All of this “guardrails” handwringing is nonsense. These things output text. Are you for censorship of a book written by a biotechnology expert that gives out the exact same information?

reply
I guess in this theoretical "AI makes weapon" scenario one could use the same AI to make defences too?

// Claude, make antiviral nanobots that defend me from 6ft virus. Make no mistakes.

reply
I don’t know if you’re being silly but it is orders of magnitudes easier to modify an existing virus to selectively target certain snps than make “antiviral nanobots”
reply
Claude, modify the existing 6ft killer virus so that it only makes my balls itch slightly for a day and gives me lifetime immunity to all further stamms of the 6ft killer virus. Make no mistakes, double check so the virus causes no unforseen complications.
reply