Well, giving it access to a simple linux terminal is theoretically enough to cause more damage than most people are comfortable with, and doing so is trivial enough that it will be done (and has been, tens of thousands of times).

Humans are biological machines that generate further humans.

Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.

We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.

I think the widespread "they are just text generators" and "they are just tools" are comforting lies rather than an honest look at what we are seeing right now. Intellectually lazy.

And by the way, there has been a long-standing consensus among ethicists, philosophers, and sociologists that technology is not value-neutral [1]. Of course Silicon Valley has a long-standing tradition of denying this.

[1] For example Footnote 1 in https://www.jstor.org/stable/27106634

or

https://plato.stanford.edu/entries/technology/#EthiTech

What if LLMs completely unrelated to the nuclear missile ecosystem autonomously hack their way in (maybe with sophisticated social engineering)?

It is a fact that among experts there is no consensus on saying '(super)intelligence is broadly safe and easy to control'. There might even be a consensus forming on the opposite claim.

Regardless, why would there be no scientific consensus if the question was easy and clear cut? I think the easiest reason is that these are hard questions to answer.

The field is much older, MIRI is ~20 years old. Look up Eliezer Yudkowsky.

Yes, this is getting ridiculous. On both OpenAI and Anthropic.

Simple example. I am a CTO, and I want to upgrade our capabilities to perform automated pentesting. We see automated attacks of growing sophistication against our infra, and I want to be able to do the same to find vulnerabilities before the bad guys do. I asked GPT 5.6 Sol and Fable to give me a summary of options. No dice, in both cases I was told I need to be an accredited researcher to get anything. A fricking summary of commercially available options is getting censored. WTF.

Meanwhile I have an uncensored qwen 3.8 27B here that will happily attempt to (as a crude and randomly chosen sampling of bad/evil things) give me the recipes for meth, how to make an IED, write a manifesto in support of a horrible ideology, or commit various forms of fraud. Now I certainly wouldn't recommend that anyone try to follow what it says to do, because it's almost certainly very wrong on key parts that would put its users in federal prison for the rest of their lives.

There's uncensored models out there which score 0 (zero refusals) on this "harmful behavior" dataset:

https://huggingface.co/datasets/mlabonne/harmful_behaviors