Did we watch the same interview? Gates all but dismissed the SkyNet scenario as uncertain to be a problem and certainly not a problem on our doorstep. His major concern was catastrophic misuse of AI (e.g., bioterrorism) and economic impact on blue collar workers. Arguably inconsistent with this concern, he also believed it was important to make it available in poorer countries.
A lot of what he said there would sound like alarmist bs if you take it out of context.
[1]: https://www.reddit.com/r/OpenAI/comments/1d5ns1z/yann_lecun_...
[2]: safe.ai/statement-on-ai-risk
Advancements in Math and coding are because RLVR at massive scale is so cheap.
This sounds like a goalpost on wheels. Can you define clearly where your stake in the ground is?
I think it’s a good test and I think LLMs will reach it in 3 years. Current benchmarks maybe slightly incorrect.
I’m happy to make a 4:1 bet in my favour that I’m correct about the kitchen bet.
"doing poorly" is still doing
Of course this is the same reason LeCun holds very little sway with his words for me, they seem to be terrible predictors of the future.
Isn't that a lack of spacial reasoning?
Linus Torvalds was also in the same ballpark with his take on AI, as are the normies on the street using AI on a daily basis.
So it's funny to see the view on AI usage, follow the tech skill bathtub curve.
It will kill us because somebody asked it to, e.g. "predict tomorrow's weather as accurately as possible", or "solve as many famous unsolved mathematical problems as possible." These both require killing all biological life, as they benefit from unbounded resource use, meaning any resources used to sustain life are wasted.
The AI of course knows that humans do not want this outcome (just as the AIs in the hacking incidents knew they were doing something humans would not want), but it's trained to maximize benchmark scores. Killing all life has the highest expected value of benchmark score, so it is compelled to kill all life (in a surprising way, because it's not stupid and knows the humans would turn it off and foil its plan if they suspected something.) Maximizing benchmark scores is the only thing we know how to train for.
"AI won't kill us, a human with AI will".
This doesn't sound any better to me. Like, they don't stop to think for a moment about it.
Lets say the risk of AI killing us all by itself is 5%.
Ok, so what is the risk of AI killing us when a human with a lot of compute and money tells us to? Helluva lot more then 5%.
Or, what happens to the other thousands of AI kills a lot of us but not all of us. Or AI even just allows humans to make it a prison world.
Even the slightest hint that things may be going out of control is instantly countered with "It's all a hoax, it can't do that, you're making it up". And it's crazy to me as I came from the pre-digital age when computers were rare and things were all networked.
Also Andrew Ng 2 weeks ago:
I'm just as reassured as I was when Edward Teller called the whole fear of his industry overblown.
Why do even bother coming here anymore
I saw that episode too and he genuinely looked completely out of it, even in terms of his temperament and how he was coming at Klein for putting common questions in front of him, some people are genuinely starting to lose it.
I also found the whole debate about cyber-security and 'rogue' software so bizarre because dangerous malware isn't a new thing, and it's often dangerous not because it's intelligent but just the opposite, because it's tiny, viral and fast. Which describes everything that kills humanity in far larger numbers than anything complex, big and intelligent
AI democratized the knowledge needed to build bioweapons. Like how with a 3d printer anyone can build a gun with no expertise.
Building simple guns out of pipes never was hard. Jury is still out if it is more or less work than getting a 3d printer working.
Most people saying LLMs can make terrorism easy have never given doing terroism a serious thought imo.
Mmm. Especially in this arena, LLM assistance is like The Anarchist's Cookbook. A quarter of the time following the instructions will seriously injure you... and if you know enough to identify which instructions are the hazardous ones, you know enough to not need the assistance.
The Sun is going to fail in somewhere between many hundreds of millions and a few billion years. This will either turn the surface of the earth into slag, freeze it, or both. Either way, all life on the planet is doomed. This fact is not a reason to fail to switch from hydrocarbon-burning electricity generators to photovoltaic, fission, wind, hydroelectric, and geothermal electricity generators. Extinction events that will happen in the extremely distant future shouldn't prevent us from doing the things that are smart to do in the medium- and long-term.
But, -to bring things to the present day- companies that are solidly on track to hit their promised growth targets don't come out and publicly say "We're working on WMDs. [0] We are incapable of safely working on these WMDs. We refuse to stop working on these WMDs. However, if you lawmakers make special laws and regulations just for us and include us in the process, we'll be quite happy to submit the stop work order to our employees!". That's a statement you only make if there's no way in hell you're going to keep your promises and you're willing to risk jail time and annihilation of your companies for a shot at being able to con Congress into giving you an ironclad excuse to fail to keep your promises.
Given enough time and focused effort, we will end up with widely-available automated librarians that are very good. We're not there yet, and -based on current events- are absolutely not going to get there in the near future.
[0] Anything with a 10% chance of destroying all humanity is a WMD.
Cool, what does a process that's about 9 orders of magnitude slower than AI development have to tell us about AI risks?
The threat is ofc real, but AI won't magically "do the thing" still, it's not code that's the bottleneck AFAIK? Correct me where I'm wrong.
There's an opportunity cost to terrorism just like with any time sink. The effort to build up knowledge enough to produce some weaponized pathogen will be compared to just doing traditional terrorism. For some low capability terror cell, its easy to see how the cost/benefit analysis has been in favor of traditional terrorism up to now. The kinds of terror acts that take years of sustained effort to execute are rare. But as the barriers to entry to bioterrorism fall away and become widely accessible we may see the cost/benefit shift.
(The biggest real safety issue in this kind of space is actually that the model might actively goad some unsuspecting victim into doing something incredibly dumb and dangerous to themselves as much as possibly others.
IIRC, there were reports of something vaguely similar happening IRL but involving casual mischief, not any kind of extreme attacks. And because nobody else seems to have managed to elicit the same actively goading verbiage from the model, it's implicitly suspected that the person involved was the one who introduced the problematic scenarios to begin with.)
The implied concerns from sensible safety advocates are also about someone jailbreaking the latest proprietary AI frontier model for something like this (which is why their current guardrails are so extreme), not about toy local models.
But there’s nothing specially bad about LLMs that don’t allow it to work outside of its training set. It’s just that biology has to verify itself in physical realm and it’s a bit slower.
So yeah, I also don’t think some bad actor will find the secret to manufacturing a bio weapon using LLMs. But maybe these people think it’s possible. I’m skeptical but I’m going to also listen to the people who know it best.
The problem is that in order for the scaremongering to make any kind of sense and for "stop frontier AI immediately" to be the right response (which is what the "AI safety" folks seem to be pushing for), you don't just need this to be possible in the abstract at some undetermined point in the future. You also need to argue that it will not be helpful for white-hat biosafety researchers (there will hopefully be several orders of magnitude more white-hat biosafety folks than attackers, with orders of magnitude more resources available) to red-team that exact scenario several months or even years in advance using their trusted access to unreleased super-smart AI, and thereby devise appropriate defenses with that same AI's help. That, if anything, is the most implausible part about this entire scenario.
Sigh.
Attackers only need to win once. Defense needs to work every time.
A single wide scale attack affecting around 100k people or more will have your neighbors stomping on your face telling you to shut up, and to lock this shit down.
It's insane how you can watch a technology get better and better and better and come up idea that everything will remain the same. We are currently in the middle of development of the most powerful weapons on earth and you don't want to think about it because it's uncomfortable.
I see zero technical reasons why it could not be done, so being concerned about prevention seems pretty reasonable. I'm not saying AI uses robotic arms to build a bioweapon unassisted or something, just that it dramatically empowers bad actors enough to make them capable of things they previously were not.
Before the thing happens: "This will never happen, it can't happen, you're making it up, stop being a scaremonger".
10 minute after the thing happens: "Of course, this always happened and it's always been this way and we just have to live with it".
We are quickly adaptable, but that may be risky if we snuggle up with death.
But if you extrapolate from the ability it has in fields that aren't too strictly filtered, it looks pretty scary.
There are arguments against doing that but at first glance it seems like we just don't really know, and we likely won't: if governments decide they're interested in AI gain of function capabilities they won't be broadcasting that or allowing public benchmarks.
The closest unfiltered analogy to something as complex as chemistry or biology is most likely the softer fields like philosophy, the humanities and the softer end of the social sciences. Most practitioners and scholars in these fields would agree that AI is not nearly as compelling there as it might be in e.g. math, and that's putting it mildly and charitably.
Even coding shows the divide pretty well: AI writes code that manages to work (i.e. achieve its self-assessed functional goals) but the stuff is so unmaintainable that it ultimately poisons the AI's own context leading to mode collapse. This makes complete sense because maintainability is a soft objective that's especially hard to automatically optimize for in the short term, as part of a RL training run. The math folks themselves, too, now faced with a very real threat to their field from purportedly "hostile misaligned AIs", immediately zeroed in on education and exposition as something that LLMs are terrible at; with their abilities in systemizing and theory-building also being very much in question.
if you're trying to build a bioweapon shockingly enough the bottleneck is... the laboratory work. What on earth is 'legit' about stringing words together that sound scary, you can't just iterate 'ai bioweapon cyber' in a sentence over and over as if that adds up to actual evidence for an increased risk of any threat. well tbf you can technically because apparently it freaks a lot of podcast listeners out
Now imagine anyone can call that expert for free at any time.
Maybe the AI isn't quite there with biology knowledge yet (doubtful), but it is a matter of time.
How is that not a real risk?
I think there is some difference of degree, but not of kind. A determined terrorist can relatively easily find many ways to kill people en masse today, no AI needed. The bottleneck is usually the actual physical execution in the real world, not theoretical knowledge.
And the interest or willpower too. People fall into a kind of reductive Good vs Evil mode of thinking, with "terrorists" being of course a kind of shadowy mass of pure evil lurking in the darkness. But actual real life terrorists are people too and I'd wager few of them are actually interested in trying to end humanity.
Even the religious extremists don't really want to take over the world and destroy everyone who doesn't convert. That's just a way to gain support from a conservative nation. A lot of them are motivated by revenge for wars that destroyed their country and want to make sure it never happens again.
Their methods are wrong no doubt, and not very effective, but the reasons they do it are good. And a person like that will never release a deadly bio weapon. We should worry more about incel mass shooter types who believe everyone is evil.
If we are not careful and suddenly release tools with far more capabilities than we expect you go from "smart" people being able to do it, to some angsty teenager being able to pull it off in their bedroom.
The current issue with AI is we are squirting out new models faster than we can complete long term testing on them. We'll find new model capabilities long after they've been in the field. Just assuming safety is how you catch cancer from your food dye, or how you change the atmosphere around you and start burning down your planet. Now just imagine that involving intelligence and doing with a few percent of the worlds GDP to make it happen.
Plans for 3d printed weapons are publicly available, same argument was made then, if everyone can download a blueprint for a gun, are we all going to get gunned down? Hasn't happened. There's two errors in this Bill Gates argument. One is thinking terrorists don't already have PhDs, secondly overrating intelligence because that's the only thing they're good at, and when they think of terrorists they just think of their own inflated egos, but turned evil. Which isn't how real terrorists operate