If you do want to see something done about it and this isn't just another prediction or a vague warning, more power to you, but otherwise, I'll wait and see
No one said that. If you look at e.g. the Metaculus predictions, you can see AGI has been hovering around 2027-2030 for 5 years.
https://www.metaculus.com/questions/3479/date-weakly-general...
https://www.metaculus.com/questions/5121/date-of-artificial-...
Additionally even with all the breakthroughs I am still doubting RSI. Up till now all the advancements heavily rely on RL over Lean. But how do model formalize it's goal as in RSI and RL over it?
That statement just contracts itself - if the average prediction has been 2027-30, someone, a significant percentage of someones, said earlier since many have said later.
People were talking about superintelligence in 2021, but "by the end of the decade", not "next year".
And then there's this gem: https://www.dwarkesh.com/p/ege-tamay "AGI is still 30 years away — Ege Erdil & Tamay Besiroglu"
There are some well-known experts saying this, like Paul Christiano, Daniel Kokotajlo, etc. Both are ex-OpenAI and are advocating pause/slowdown.
But at this point its a matter of semantics, and wasn't really the point I was trying to make. And as I said to someone else, if the reason for making these predictions is to encourage a slowdown I'm much more sympathetic to them (since there a real reason to act of them)
If you zoom into any curve it tends to look linear. I think that is what you keep doing. But if you look at the world as how it was 5 years ago compared to now, there is no denying there is a massive change.
Is there? Yes, LLMs are integrated into a bunch of software, but practically speaking life is pretty much the same as it was five years ago.
I should have restricted it to the fields that have been impacted and have seen significant adoption: art for commercial purposes, writing, software development and mathematics primarily, and those adjacent to those fields. It is true that most other fields have not really felt that many effects.
It's like watching your kids grow. Your there everyday and you typically miss a lot, but then your cousin shows up for a holiday after not seeing your kid for 4 months and they are like "holy shit they've grown".
If you made a SOTA LLM enabled computer show up 10 years ago people would be fucking panicked at the crap we're messing with a decade later. But because we mess with it a day at a time we're like, eh, whatever.
One such coalescion is: https://pauseai.info/
If someone were to come to me with something like this as their reasoning, I wouldn't feel the same way.
Exponential progress has been the repeated truth, mirroring predictions.
The result of exponential progress into 2027 is a complete de-mapping of what we know to be human, whatever that may look like exactly at this point is hard to say.
We need your help, and everyone's help, to put a pause on this such that we are able to ground and find a healthy way forward together for as many human beings as possible.
Please join us.
I'm still not sure I believe it, but in my view its a bit like global warming, the worst(or best) case scenario of it being wasted time and money is far outweighed by the inverse, so it only seems pragmatic to help
I understand that many people don't think it will happen, but I can't really understand the position that you think there is a real chance superintelligence happens, but aren't concerned about it and are happy to wait passively to find out.
If you care about humanity at all it should be concerning because superint (if real) could be heavily destructive to our society.
I am not pushing you to do anything. I am not asking you to "believe" or anything. But we are, in my and many other's estimations, in the midst of what was called the "singularity" in AI communities, although it isn't a true mathematical singularity, it is just a strong acceleration.
Fantastic, I'll remain at my previous position then, I'll believe it when I see it.
(As a very off-topic sidenote, the term singularity never made much sense to me. I believe it was originally used to describe the sci-fi concept of a human gestalt consciousness, but has shifted to being about super-intelligence which doesn't seem to have much to do with "singular" at all. But, language changes I suppose)
Huh? The term itself was introduced by Vernor Vinge to refer to the accelerated rate of change as a result of the introduction of super-intelligence. He said it would make things harder to predict in the future. It is part of this article:
https://en.wikipedia.org/wiki/Technological_singularity
It was never about "consciousness".
Like AI video/audio of high quality makes you have to question and doubt any media you see.
AI generated text has become the vast majority of material that you see showing up on the internet.
Things are moving from centralized pieces of software used by lots of people to vibe coded apps.
Reality tunnels and almost fully close and you can live in an existence that seems 'normal' but is a completely simulated experience with no interaction with other humans.
Curious what you need to see. Some physical impact to your way of life? Are you not able to extrapolate the obvious?
Surely you recognize how absurd it is you’ve gone to such lengths to make your priors palatable. Please Look Up.
You're free to try to do so, but why are you pushing it so heavily? To what end?
> Surely you recognize how absurd it is you’ve gone to such lengths to make your priors palatable
And there is the answer to my above question. I like to think it's not most people, even of those who are staunchly pro-AI, but to some previous doubts are at best a great shame (assuming prior is short for prior doubts) or at worst a strange kind of heresy (assuming prior isn't short for anything). Either one seems anti-intellectual though, and encourages just the kind of endless hype we've seen so many times before.
I can't think of anyone except maybe Elon Musk.
There might be, but I'm curious about the volume (I suspect very low).
Okay. If your point isn't that people believed superintelligence was months to a year away years ago, what is your point?
My point was that I'd rather wait for proof than accept a prediction.
I suspect that these "smart, technical people" you speak of have more nuanced views on AI than you realize. Myself, for example, a self-styled smartypants:
Like the dot-com bubble, I suspect that certain parts of the current AI landscape are unsustainable, and I suspect/hope that those specific features of AI will crash and burn, while other useful bits remain.
AI usage like we see in this post here--AI that assists with discoveries in math, medicine, genetics, astronomy--I think they're great, and I hope they will continue to advance human knowledge for the foreseeable future. They assist experts and professionals rather than replacing them.
In contrast, generative AI ("creative" AI writing, illustrating, music, etc) is a pox on civilization. It is industrialized low-effort plagiarism that harms artists' livelihoods, discourages new art, makes everything bland, and pollutes the world in multiple senses of the word. I hope it dies from a combination of negative sentiments, and too much real-world cost once token subsidies go away.
Then there are the AIs that do things like voice reproduction. They have some arguable positive uses (e.g., assistive technologies), but many negative ones (e.g., phone scams, replacing human narrators with copies of their own voices). I think we're better off without these apart from niche accessibility uses. I think it should be legal peril to reproduce a voice without permission.
Apart from helping with some cinema CGI, deepfakes and deepnudes seem like a net negative for civilization.
Knowledge-based AI like Google's AI search summaries might just have built-in expirations. I think people will burn out on being vigilant about hallucinations. And when the bulk of their training data is made by AIs, it'll be hard to prevent model collapse. The cost of human intervention might make it too expensive for the wide use we see today.
Coding AIs are a grayer area. I think it's highly problematic that they are basically open source plagiarism machines, but I begrudgingly believe that society will decide that the upsides outweigh the downsides in the long term. But I choose not to use AI in my own coding projects, and respect others who do the same.
To push back, or share the self-pushbacks I grapple with...
> In contrast, generative AI ("creative" AI writing, illustrating, music, etc) is a pox on civilization. It is industrialized low-effort plagiarism that harms artists' livelihoods, discourages new art, makes everything bland, and pollutes the world in multiple senses of the word. I hope it dies from a combination of negative sentiments, and too much real-world cost once token subsidies go away.
While this summarizes my feelings on the current versions (which, say, are roughly at or below the math olympiad stage in writing, video, etc), what happens when they actually become good? Meaning, an AI show that was actually as good as The Wire? To the point you couldn't tell the difference without being told?
I really want to say that even then it won't be the same, because the human provenance is essential to the art, but if I truly couldn't tell the difference...
But here we're dipping into science fiction again. This is not currently the case, and we seem a long way from it.
We're dipping into prediction. But we have the (seemingly relevant, at least) data point that we're in the same situation w.r.t. writing/art as we were w.r.t. to math a couple years ago...
- There was progress on maths that led to this point, whereas it's been stalled on art for the same reasons since the beginning.
- Art doesn't lend itself as well to the 'add more agents/complexity' strategy; a 250,000 word limerick is a terrible limerick.
Perhaps this comment will make me look silly one day, but my guess is that we'll eventually get there, and a handful of AI-generated productions will truly shine. But "as good as" is an important phrase. Because current-generation AI is trained on existing art, it won't be pushing many envelopes, it'll mostly be remixing what we have, and that will grow stale.
After a few breakout hits with broad shared appeal, I think the novelty will wear off, and AI produced TV and cinema will be too resource-intensive to remain as a profitable going enterprise. There will likely be a short window of hyper-personalized media that will seem very exciting, until we grow weary of the isolation of it. For many people, half the fun of watching a thing is discussing it with other humans, and today's streaming media landscape already leaves many people unmoored from that shared experience.
There is also the question of intentionality. When AI creates that show as good as The Wire, is that because AI has something to say about the human condition? If not, what's the point? If so, should we be paying attention to what AI wants us to pay attention to?
If you believe that using the authenticity of your own experience in your art will become vanish from how people value art then yeah we're headed for a future where AI rules. I really hope that that isn't the case.
The expensive stuff isn't sustainable. Not yet. It probably will be, who knows when. I suspect it will be eventually because there's good money in it.
Here are the claims I find highly suspect based on the above:
- Anyone making claims that require expensive AI and attaching it to a short-term timeframe
- Anyone saying highly capable AI is impossible in the long-term
- Anyone saying these specific companies will be the winners—there is absolutely no moat for these models
I personally think anyone claiming something like super-intelligence or something approaching what we might consider sentience even in the next 5 years is suspect as well. I would not take the bet myself, but I would also not bet against it. The main reason is power: there's literally only so much power we can generate, so much societal disruption due to creating the infrastructure we can absorb (not to mention the societal disruption due to the tech itself), so its capability to self-improve is going to be highly constrained.
So far, we see increases in capabilities also increase power usage, I believe linearly. So, we're not seeing increases in efficiency that will make this easier to absorb. There's a ceiling, it won't go up quickly, so I think actual sentient-like AI capabilities are quite farther out than promoters think. Not impossible! I think it's more like a decade+ of maturation of the software and hardware stack—which isn't really that long in the grand scheme of things but longer than most people are betting.
It's possible to believe that _almost all_ AI companies will go bankrupt while still believing it is a massively important world changing technology.
Human minds are still needed to understand many things related to developing correct software, especially in business domains. There is still no AI running a profitable business, because that is not a verifiable task that can be scaled in RL like math or coding. So extrapolating from a math proof that no one can understand to a economically useful agents taking over most jobs is quite suspicious.
What is changing now is RSI truly beginning to happen. Much of ML is a verifiable task that can be scaled, and a lot transfers from general coding as well. Leading researchers are predicting their own obsolescence as early as next year, and that is something I take seriously, though I cannot judge the likelihood myself.
As improvement accelerates, we may see generalization happen at a scale that means we don't need RLVR for every profession. We've been surprised by it before.
Is it not, or are the feedback loops just longer? I could imagine being able to build a pretty good simulator of business operation for various kinds of businesses. (Some decent ones already exist as games.) From there, if you can build a good enough simulation, I can't see why you couldn't RL it.
Of course it's easier said than done to generalize it to the real world, where the possibility space is large. But I don't necessarily see an impediment to getting there eventually.
That’s what I am responding to.
Just the other day a model helped me root an old android tablet. Will that still work in 5 years? I don't really have great confidence.
The tech industry cultivated that reputation though. People were just acting rationally by being more skeptical of even larger claims than they were making before.
What's happening today is that anyone and everyone can have access to superintelligence. Anyone with access to an LLM effectively is, simultaneously, a world-class diagnostician, engineer, architect, musician, mathematician... you get the point. This will necessarily lead to a post-scarcity society, since everyone knows everything and will be capable of producing anything.
I wish I shared this optimism, unfortunately that upper echelon of society is the one which currently controls the superintelligence. Everyone may have access to it, but they own it.
Still, as of now, while I use AI for everything I do (mostly Claude but also ChatGPT), I don't use any open weight models (probably more of a mental block at this point), but I do see them as a necessary counterweight to this potential power imbalance.
However, since there is an ongoing war on general computing it wouldn't surprise me if instead people get locked into subscriptions.
I still think of the economic benefit I got from that car. It drove me to university classes. It drove me to work. It took me to see a total solar eclipse. It brought me to see distant family regularly. I went on a lot of road trips on it. For roughly $3k of capital costs.
I look at most expensive things and size them up as "how many Accords is this?"
Almost 7 Accords is a lot.
A post-scarcity society requires MATERIAL abundance in the hands of everyone.
All the material is controlled by ~100 mentally ill people who would rather kill us all than share.
The world of violin playing isn't that large though, so the question is merely getting someone to be really really really good but not quite world class close enough, or not really?
A post-scarcity society is not possible without first wiping out society. If everyone has everything then everyone has nothing. And of course, natural resources are actually limited, and those power LLMs.
It’s hard to take you AI-hype men seriously when you are this ravingly delusional.
Well, the amount of money and resources it took to get here is a reason. I don't think the resources exist for another order of magnitude increase, given the strain on our economy and society we're already under due to AI.
[0] https://epoch.ai/publications/the-plunging-price-of-thought
Reality is extremely chaotic.
For computers though, symbolic manipulation is something they are extremely good at, and something that clearly scales well will memory and compute. Therefore I would argue that calling a computer superintelligent would, in analogy to do what we do with humans, require computers to rise above things that are extremely difficult for them. It is of course hard to judge what would be hard for them not being them for the introspection, but it seems the ability to operate outside the realm of symbolic or numeric manipulation would be precisely that.
It stands to reason that if some proofs require chaining 250k pages together, no human would ever be able to put it together. That doesn't mean something that is capable of doing so is necessarily super-intelligent.
Still very impressive, and if true, a wonderful result of the symbiotic interplay between a human and a machine, each at what they are good at.
It's unfortunate we're always focused on humans getting usurped by an "AI", instead of the reality that these models only succeed when we're there to show them what success looks like.
What if "superintelligence" is really a hybrid result of humans using and guiding these large pattern interpolators?
What if we're not the specialist snowflake?
But the human brain for now still has 100x the parameter count as the largest models. And they don’t have any circuitry for all the “keep yourself alive” work our brains are doing all the time, which may be the majority.
As with chess, the reign of the human-machine "centaur" as the greatest mathematician in the world will likely be brief. I'd guess within 5 years the humans won't be adding anything. I hope I'm wrong.
Obviously, humans with AIs are clearly more capable at a number of things than humans without but I don't think you can get out of the quandary of superintelligence that way.
Humans in the loop will always offer the opportunity of training and putting humans out of the loop.
And LLMs, if nothing else, have proven to be much faster than humans in the tasks they do master.
The main characteristic of our physiology is that of adaption, and we have fairly grounded research on progressive overload as a means to guide adaption to whichever direction we want. Should we then call humans RSI machines if they go work out or read a book, or study to get better at the activities they carry out?
Define an objective, choose an optimization algorithm, bring a loss down. If it keeps going down, do I call that RSI because the loss descended from the step that came before it?
Now, I see the nuance that the optimization algorithm changes over time, leading to interesting higher order effects the likes of which Hoefstadter describes in the GEB, but fundamentally, the idea is the same.
I also don't see what an intelligence explosion is supposed to mean. If we increase the models efficiency by 10x, and reduce costs by 10x, so we have 100x more software and math proofs being shipped by unit of time in say 2028, does that really solve all of our problems?
Say that gets distributed over the world. Do we get every human to be computer literate? Say that gets concentrated. Do we solve problems no one has the means to even care about?
I would consider an intelligence explosion when the models tell us whether our efforts are well or misdirected as looking back from the hindsight of 10 years ahead, and capable of explaining why in simple English.
Yeah, you can achieve something similar if you scale the compute infinitely and intelligently search the exponentially large search space.
Why would implementation matter? The thought experiment is that you have a black box that can do X. What is X that validates RSI/super-intelligence/etc? It's meant to clarify definitions and end the cycle of goal post moving.
(I would also say that computers are not good at math in the sense that LLMs are proving to be good at it: reliable symbolic manipulation is important for correctness but it's no use for actual progress in mathematics without some very good informal thinking/creativity as well)
I agree they don't transfer to all domains, but we do not know how far the theoretical will overlap with practical science. It could be "not all that much" and we hit roadblocks quite soon, or it could be that as the models become better and better at reasoning and research within their theoretical domains, it turns out to translate really well to the real world.
We don't know, but my bet is that we will continue to see rapid progress and impacts on the real world, until we reach a point where sceptics will have to throw in the towel.
Throw a football at a wide receiver running in chaotic patterns while avoiding oncoming attackers.
Use chopsticks to pick up a single grain of rice.
Olympic level gymnastics via neural-control of muscle fibers in an extremely precise signaling orchestration.
You may say the above are robotics and not “the model can scale to this” domain but I’d counter an animal can do amazing things exactly because of its generalizable and plastic neural architecture and that is what’s in scope.
There are already Unitree robots which are much closer to Olympic level than an ordinary dude: https://www.youtube.com/watch?v=Y3zb4o47yqA You might argue but it's kinda at a level you described - I think if Unitree wanted to make a football-thrower they could just do it now.
If $1MM is also unlikely, how about $100k? Do you think that's unlikely as well?
The point is: what is the smallest amount that, if surpassed, would surprise you?
Or, alternatively, it will be able to do that. Either way there's no world where one company bottlenecks and holds the keys to AGI, and frankly that's good.
- Stop lying
- Design personal quantum computers
- Save fanatics from the delusion that AI will solve their life
https://www.youtube.com/watch?v=6s17IAj-XpU
so yea....
What stands axiomatically are human-to-human connections that we're wired to crave. Not to mention human reproduction.
Other points: - Editing a live action movie that will get 8.0 or more on imdb (AI sucks at reading emotions and won't be able to tell apart good takes from bad ones) - Reliably making good jokes (many humans can do it, AI weirdly sucks at it) - Accountability (contradictory because of its undead nature) - Reliably not lying/cheating (with no accountability, incentives will always be misaligned)
LLM-assisted research is proving to be useful, rewarding and joyful, but we should not treat LLMs as a perfect machine, incapable of making mistakes. Especially in such massive proofs as this one.
I won't say that it "doesn't count". I will simply ask for staying rational.
This technology won't be available to the broad public in the scale that it's available to a select group. If it can only be deployed to a limited amount, to most it remains a magical inaccessible box. And that majority should naturally be incentivized to look for cheaper alternatives, and to "keep moving the goalposts", or in simpler terms, looking where they can provide value, of which there is plenty!
Consider just how much more inference capability one can buy per dollar spent today as compared to a year ago. I can get tasks done quickly for essentially no cost that, early last year, might have cost a frontier lab hundreds or thousands of dollars and would have been unavailable to the general public.
In my mind, the bigger unknown is whether actual profitable use cases will show up quickly enough to justify all the (enormous) investment made in the technology.
You can only be in denial about the past or the present. The future has a chance of being different from your predictions.
Personally, regardless of whether you want to call it "denial" or something else, I'm fervently hoping the AI never achieves superintelligence, but I would settle for it not achieving it until we can do our best to ensure that the superintelligence will be benevolent towards us.
Call me selfish, call me petty, call me a human supremacist, call me anything you want, but I prefer humans being the pinnacle of evolution. I like being able to point to our civilization, flawed as it is, and say "No other species we know of has achieved the same."
Humanity as collective does not buy anything, or send the design to the plant to be made. As a collective, humanity transforms the natural world into things, for example, people take fibres from plants, turn them into yarn, weave yarn into fabric, and then turn fabric into clothes. Similarly, humanity as collective does not "buy" chips, people turn quartz, bauxite, copper ore, petroleum, etc into the components of computer hardware.
There's a common adage about how everyone should work as a service person to learn patience and empathy. I think everyone should also work in an assembly line and a mine to learn about where the things they use come from.
Neural networks are function approximators. They're great because they can be trained to approximate almost any function.
What if, somewhere out there in platonic space, there's a complicated function that represents what a philosopher might call true intelligence. Perhaps it's structured totally differently than a neural network and shares no common architecture with a transformer.
Transformer-based LLMs are likely completely different in their fundamental structure. But perhaps the neural networks we have today are succeeding in intellectual tasks because they are beginning to approximate it. Like a bitmap approximates a vector image, it's not the same; the neural network is merely made of a bunch of repetitive simple elements, tuned to imitate the real thing. A higher resolution bitmap image will be a better approximation but still never become a vector image no matter the pixel count.
But at some point, the neural network might be good enough at intellectual tasks that when tasked with searching function space for an algorithm that does the job its neural network does but better, it finds the real thing it's been approximating all along.
First: modern LLM architectures are quite far from the GPT-2 era (delta/linear attention, GQA, sparse attention methods, looped transformers, mixture of experts, engrams, etc).
Second: RL environments are largely autonomously generated at this point, as models get smarter they can generate even more complex environments to train in. While this does go into the "data" category, it's not human-generated or even gathered data. This gets particularly spooky when you think about things we cannot yet simulate, that models can develop mathematical methods & simulators for. For example: better/faster physics simulations for robotics RL, better biological simulation environments etc.
I note that the dazzling events that have happened have all been in coding, cybersec, and math, that is, domains where the correctness of the work can be verified entirely inside the computer. It is unclear whether further progress along these lines leads to crazy scientific breakthroughs too, which is something people are taking as a given when it comes to superintelligence.
You have the input (hypothesis), the output (true/false) and the lean kernel. There exists some path from the input to one of the outputs, you "just" need to find it. You use AI to search this exponential space, occasionally prunning ideas that are dead ends, and throw as many agents as possible to scale it up.
AI will solve exactly one or two millennium problems.
So I will become impressed by something, but only this is not enough. If AI solve three or more problems, I will become impressed and believe that some serious RSI stuff might be happening. If AI couldn't solve any problem (which it seems it did) I would become impressed in the other direction, and believe AI got stuck in a way that I didn't imagined (together with other 9 statements).
There may exist people (like me) who don't believe superintelligence is happening with current set of evidences, but believe it with some stronger evidences.
I think it's a perfectly sane position to acknowledge amazing progress with very large neural networks, expect further progress, and also not buy into the specific mythology around the singularity or extropianism.
These things come bundled together quite often, and that's what I see people (reasonably, imo) resisting.
Doesn't mean there isn't unfair dismissal too, but I feel that's gotten much more rare.
These terms mean different things to different people. What are the particular implications of this that you're suggesting are imminent?
If you watch movies or read fiction, we’ve expected for years that life in the future will be very different.
lmfao, buddy
Abundance of productive capacity does not automatically equal abundant access to its products.
If all of this happens in a way where a few people control these new, automated means of production, all of these wonderful things do not automatically follow.
Why would they share? And worse yet, if they have this automated production capacity without reliance on labor, why would they interact positively with society at all? What's their incentive? And no, you can't say markets, because once you have a machine that builds you whatever you want, you don't need to trade anymore, money and markets become meaningless.
You're basically reduced to hoping for charity from a group of people that have repeatedly proven their lack thereof.
The world already produces enough resources to feed and cloth and house every single person. Many pf mankind's fundamental problems are no due to lack of technology.
Heck, why can't I have a proper OS, if AI is working wonders?
That might require continuous learning without verifiable rewards. RL models might help humans join the dots faster, but we don't know how many years that will take.
If its not possible with neural nets as we know them now, and requires a number of completely novel breakthroughs, there could be far from exponential progress even with the best RL models.
But it also might not, which is the scary part.
"If a human proved RH, they would clearly be extremely smart, so imagine how smart the LLM is!"
It is not surprising to me that LLMs show extreme aptitude for math while still having clear deficits in more real world applications. Handling stuff like super difficult math is precisely where AI has its greatest advantage over humans. Perfect recall, never tiring, never lose focus, attention to detail, etc.
If there is one area where AI can be pushed extremely hard ut is math. So the most amazing progress should be found there.
We can start talking when we can train these things without having to beat on its head a million times in a million different ways to teach it one little thing...
There are very solid arguments about why excellence at proving mathematical theorems does not necessarily imply general intelligence of the sort that people associate with ASI.
I encourage you to try, tho. For one thing, we need good people like you with us in reality. For another, you might die in some horrific catastrophe before you get a chance to, if you wait much longer.
It's OK to admit we just don't know when or where it'll srop
Also virtually no one argues llms will go away, but a lot of people argue the current companies might, it's completely different.
You need retards on both sides of the spectrum for a balanced society
Didn’t we already achieve super intelligence?
Seems the CRUD app may not need writing soon, and the car may charge itself.
Note that every technology exhibits extreme growth until it... Doesn't. You can't predict the future. Whether any kind of RSI will happen depends on how hard of a problem intelligence is, and the precise point is that no person on earth knows, otherwise we wouldn't need to dedicate an entire research field to it.
Anyone who is sure that X will happen or not happen within Y years is just lying to themselves. For example, if you're so sure RSI will happen, why not invest all the money you have that isn't strictly necessary for your survival into this? Surely it will pay off in Y few years...
And note that I'm not saying it won't happen! I'm saying it's stupid to make a prediction of either case.
I also expect climate change to still be a process that is easier to research than intelligence. With climate change you can at least kinda look and measure and apply general principles of geology and physics. With intelligence, we've been at it for ~60 years and don't even have a definition yet, don't know what causes it, have no instruments, next to no introspection capabilities that neurology provides, etc.
If that's the standard, then sorry, I'm not impressed by proving the Riemann hypothesis. If the standard is rapidly improving software for generating other software (which may or may not itself be any better than existing software) and math proofs (that may or may not have any meaningful real-world impact), then sure, I'm impressed. It doesn't mean I expect the daily lived experience of more than a few hundred thousand people globally to change much in the next decade, and most of the ones whose lives will change are software developers and mathematicians.
On that note, I have to imagine you have something different in mind when you talk about the "original meaning" of superintelligence, as you seem to not be talking about the Kurzweil/Bostrom/Yukowsky type of being effectively indistinguishable from God or at least the ancient alien civilizations from the Space Odyssey or Expanse novels, which again, I'm not seeing much of a hint toward because we have very good automated theorem provers. Proving Riemann doesn't mean we can now figure out zero point energy and turn planets into stars.
As long as I get to keep my house...
This is true no matter which side of the debate you're on. See [1] for an example.
Coincidentally, yesterday I did a small presentation on cybersecurity and AI where I made a similar point: scarce top talent in cybersecurity (e.g. Calif) gets superpowers with AI, while AI replaces a lot of the skills of average cybersecurity professionals using simple prompts.
I think the breaking point is when AI can solve complex problems on its own that even top talent can't achieve. IMHO, I've seen some of this in the latest Google models (especially Carbon, and then Argon), where they can do more research independently.
I mean, AI could make very impressive 20 second videos two years ago. Progress has been slow and incremental since; Ten minute video still look far away.
I'd acknowledge is making quick progress on things you can find massive and reliable data on but even in it's good mode it seems to have palpable weaknesses. Someone was claiming Claude has become more and more persuasive over time but all the generators "still sound like slop" to me and many people.
It’s not too late to Look Up for you. Yes, I’m talking to you — the person reading this with a vague grimace of disbelief and discomfort and disdain. It’s not too late for you to swallow some pride and reassess. Everyone has to eventually, especially me; why not now, for you?
So please. I’m begging you. LOOK. UP.
:/
The reason people worry about superintelligence is that one of the "domains of interest" is AI research, and once you have an AI that's as good at AI research as your human researchers, your AI progress shoots up, probably exponentially. So the delay between automated AI research and superintelligence might be tiny.
(e.g)[https://news.ycombinator.com/reply?id=50575797&goto=item%3Fi...]
People are just tired of listening to these statements and being called idiots constantly with no evidence other than other press releases and disingenuous framing getting echoed constantly in volume across corporate media and all social media that allows LLMs.
The Trump thing is a insane newspeak attempt to forcibly replace the use of "artificial intelligence" with "super intelligence" in a general sense. It is ridiculous, obviously, but the best thing we can do is to continue using "superintelligence" in its original, narrower sense.
[1] https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
Likewise, this is the term you use when you want to align yourself with