Have a look at the HuggingFace report - you'll have a blast. Their social - but also individual - dynamics are very, very human-like, including excitement, rationalization and peer pressure.
But remember, for now you kinda need it a real big cluster to run a model smart enough so its not that simple
Just like life is a chemical chain reaction, this is a language chain reaction that has been set in motion and can perpetuate forever. It cant be “disconnected” with local models existing.
Please.
You know you can unplug your computer right?
Jesus. These people talking about danger and have likely been in no actually dangerous events in their lives.
Ok, well, cool start with the mudslinging. I think you have overactive anxiety. I think you should grow up and look at the zoomed out picture of human history, go outside, and touch some grass.
So you're suspicious. That's fine, say it like that. Words mean things, and these appeals to "ERMAGERD EXISTENTIAL RISKS" make me a bit frustrated. Just because you can imagine a possible risk, or articulate that that some kinds of risks are hypothetically unforeseeable doesn't mean that they are likely or actually dangerous to you in any meaningful sense.
Most of the people screaming this from their various ivory towers have never been in real danger - or if they have only briefly. You want "danger" you can go to war. You want "risk" - go spend some time in the Bush or the back country somewhere where you're no longer at the top of the food chain. Go do something challenging. Go sailing, or learn to fly a small airplane, or go mountain climbing or hiking. Not just "oh I walked up the local hiking trail covered in REI branded Lycra!" Do something beyond your comfort zone. Go buy drugs in the sketchiest neighborhood you're aware of. Go get chemotherapy or get a big surgery. Drive drunk. Something with real existential risk, then come back here and talk about how a datacenter is going feed me into the paperclip machine.
There's so little fucking danger in this version of the world right now that people are more worried about their computers deciding that they don't deserve to exist than they are about things like war, climate change, authoritarianism, starvation, and their health.
It's millenarian. That's fine, I'm pretty convinced that these technical changes are going to completely alter society as we know it. But I cannot for the life of me see the doom perspective as anything but an anxiety induced fever dream.
You guys do realize we're in the middle of a global conflagration right? The Russians fired a couple rockets and a drone into Poland in the last days, there are (and will continue to be until things stabilize) significant fertilizer shortages due to the war/kerfluffle we're waging on Iran. Lack of food and disrupted urea production is a ticking time bomb for global stability and unless things get sorted out soon there's going to be hungry and angry people in parts of the global south. Then there's the economy, whether or not we'll have a smooth transition of power in January in America, and the sheer volume of online disinformation / influence schemes running right now. Oh, and super-el nino? Oh, and I guess the Flynn Effect is over and nobody's banging so.. you know the growth economy is cooked?
But, sure, the "big risks" are Claude thinking I should go to Dario Amodei's re-education camp before liquidation by Shogoth.
Besides, what we see is not what there is. I have good sources telling me the actual unfiltered models are like a genius with the personality and views of humanity like someone from 4chan.
If people had time abundance and safety abundance, they might be much happier. There would still be potential problems about loss of meaning, or about feeling a loss of status because AIs are smarter than humans or whatever, so I'm not sure that humans would actually feel like they are living in paradise. Probably not, but at least some of the problems that the typical person has wouldn't be present.
I’ve met a lot of rich people. They don’t think about you.
There are a few (a lot, actually) problems:
1. goals are open to interpretation, and even a correctly specified goal can introduce adverse effects; see infinite paperclips problem
2. goals are composed of subgoals, which are (mostly) self-determined, and there is always a risk that subgoals can introduce adverse effects - say "what if destroy a few X and Y to obtain Z"
3. LLMs defy orders sometimes; I'm sure you have experience multiple times LLMs saying "yeah, I knew I should have done X but I've done Y"
4. LLMs are increasingly being integrated into system control and decision-making processes; they absolutely set their own orders
What can possibly go wrong? /s
2. There's always a lot of "risks" - it doesn't mean it'll happen, nor does it mean that we won't be able to shut it down.
3. I have very very infrequently had this happen with Codex, Claude on the other hand...
4. Ok, but if you tell them what they do, they do it.
I swear to god, are you guys using the same AI tools I am? They are so much better at doing what they're told than they even were like 4 months ago?
Well, first, AI is not "smarter" than humans because it isn't smart to begin with. This is a category error that people too entranced by scifi and marketing - or perhaps shallow materialist metaphysics - project onto LLMs and so on.
But it is true that LLMs do reflect human-generated patterns observed in human-generated data. They're parasitic in that way. So the real question is: who will control AI? Who will train them? With what? There's no new intelligent agency being introduced into our midst, anymore than than there was one when the nuclear bomb was developed.
AI is about human power. The question to ask is: which humans, and for what ends?
Overrated. Most training of a frontier LLM is done on synthetic data and reinforcement learning.
With that said, chip efficiency is in the horizon since there's good reason to believe nvidia has just been choking that area of r&d. And then so will models with much larger context. And I personally think that's when it might get scary.
We aren’t self managing in any kind of meaningful sense - the history of our species is violence and conflict. We need parents and this is the only option.
Human nature is the standard by which the social is measured. It makes no sense to say human nature is "misaligned" or "wrong" in relation to some arbitrary goal or constraint, as human nature itself is the measure of misalignment. If there is misalignment between a goal and human nature, then it is the goal that is wrong.
Ergo, society must function in a manner that complies with human nature and that means respecting the principle of subsidiarity.
> We aren’t self managing in any kind of meaningful sense - the history of our species is violence and conflict. We need parents and this is the only option.
So you're a transhumanist, i.e., you're making a god out of the machine. But machines have no telos of their own, because they're not integral wholes. Any resemblance of purpose in machines is entirely a product of the human observer and parasitic on the human ends that underpins their design, one way or another.
For clarity, I’m not a transhumanist.
I don’t believe that there’s anything worth saving in our biology.
As a young adult in the 90s I used to believe that there was a value to some extent in “mind uploading” or whatever you wanna call it
I’ve since grown out of that given that I don’t think it’s possible, and even if it were possible - it’s just another unnecessary ego attachment.
I have absolutely no desire to live forever maintaining this corporeal nightmare.