I suspect that these "smart, technical people" you speak of have more nuanced views on AI than you realize. Myself, for example, a self-styled smartypants:
Like the dot-com bubble, I suspect that certain parts of the current AI landscape are unsustainable, and I suspect/hope that those specific features of AI will crash and burn, while other useful bits remain.
AI usage like we see in this post here--AI that assists with discoveries in math, medicine, genetics, astronomy--I think they're great, and I hope they will continue to advance human knowledge for the foreseeable future. They assist experts and professionals rather than replacing them.
In contrast, generative AI ("creative" AI writing, illustrating, music, etc) is a pox on civilization. It is industrialized low-effort plagiarism that harms artists' livelihoods, discourages new art, makes everything bland, and pollutes the world in multiple senses of the word. I hope it dies from a combination of negative sentiments, and too much real-world cost once token subsidies go away.
Then there are the AIs that do things like voice reproduction. They have some arguable positive uses (e.g., assistive technologies), but many negative ones (e.g., phone scams, replacing human narrators with copies of their own voices). I think we're better off without these apart from niche accessibility uses. I think it should be legal peril to reproduce a voice without permission.
Apart from helping with some cinema CGI, deepfakes and deepnudes seem like a net negative for civilization.
Knowledge-based AI like Google's AI search summaries might just have built-in expirations. I think people will burn out on being vigilant about hallucinations. And when the bulk of their training data is made by AIs, it'll be hard to prevent model collapse. The cost of human intervention might make it too expensive for the wide use we see today.
Coding AIs are a grayer area. I think it's highly problematic that they are basically open source plagiarism machines, but I begrudgingly believe that society will decide that the upsides outweigh the downsides in the long term. But I choose not to use AI in my own coding projects, and respect others who do the same.
To push back, or share the self-pushbacks I grapple with...
> In contrast, generative AI ("creative" AI writing, illustrating, music, etc) is a pox on civilization. It is industrialized low-effort plagiarism that harms artists' livelihoods, discourages new art, makes everything bland, and pollutes the world in multiple senses of the word. I hope it dies from a combination of negative sentiments, and too much real-world cost once token subsidies go away.
While this summarizes my feelings on the current versions (which, say, are roughly at or below the math olympiad stage in writing, video, etc), what happens when they actually become good? Meaning, an AI show that was actually as good as The Wire? To the point you couldn't tell the difference without being told?
I really want to say that even then it won't be the same, because the human provenance is essential to the art, but if I truly couldn't tell the difference...
But here we're dipping into science fiction again. This is not currently the case, and we seem a long way from it.
We're dipping into prediction. But we have the (seemingly relevant, at least) data point that we're in the same situation w.r.t. writing/art as we were w.r.t. to math a couple years ago...
- There was progress on maths that led to this point, whereas it's been stalled on art for the same reasons since the beginning.
- Art doesn't lend itself as well to the 'add more agents/complexity' strategy; a 250,000 word limerick is a terrible limerick.
Perhaps this comment will make me look silly one day, but my guess is that we'll eventually get there, and a handful of AI-generated productions will truly shine. But "as good as" is an important phrase. Because current-generation AI is trained on existing art, it won't be pushing many envelopes, it'll mostly be remixing what we have, and that will grow stale.
After a few breakout hits with broad shared appeal, I think the novelty will wear off, and AI produced TV and cinema will be too resource-intensive to remain as a profitable going enterprise. There will likely be a short window of hyper-personalized media that will seem very exciting, until we grow weary of the isolation of it. For many people, half the fun of watching a thing is discussing it with other humans, and today's streaming media landscape already leaves many people unmoored from that shared experience.
There is also the question of intentionality. When AI creates that show as good as The Wire, is that because AI has something to say about the human condition? If not, what's the point? If so, should we be paying attention to what AI wants us to pay attention to?
If you believe that using the authenticity of your own experience in your art will become vanish from how people value art then yeah we're headed for a future where AI rules. I really hope that that isn't the case.
The expensive stuff isn't sustainable. Not yet. It probably will be, who knows when. I suspect it will be eventually because there's good money in it.
Here are the claims I find highly suspect based on the above:
- Anyone making claims that require expensive AI and attaching it to a short-term timeframe
- Anyone saying highly capable AI is impossible in the long-term
- Anyone saying these specific companies will be the winners—there is absolutely no moat for these models
I personally think anyone claiming something like super-intelligence or something approaching what we might consider sentience even in the next 5 years is suspect as well. I would not take the bet myself, but I would also not bet against it. The main reason is power: there's literally only so much power we can generate, so much societal disruption due to creating the infrastructure we can absorb (not to mention the societal disruption due to the tech itself), so its capability to self-improve is going to be highly constrained.
So far, we see increases in capabilities also increase power usage, I believe linearly. So, we're not seeing increases in efficiency that will make this easier to absorb. There's a ceiling, it won't go up quickly, so I think actual sentient-like AI capabilities are quite farther out than promoters think. Not impossible! I think it's more like a decade+ of maturation of the software and hardware stack—which isn't really that long in the grand scheme of things but longer than most people are betting.
It's possible to believe that _almost all_ AI companies will go bankrupt while still believing it is a massively important world changing technology.
Human minds are still needed to understand many things related to developing correct software, especially in business domains. There is still no AI running a profitable business, because that is not a verifiable task that can be scaled in RL like math or coding. So extrapolating from a math proof that no one can understand to a economically useful agents taking over most jobs is quite suspicious.
What is changing now is RSI truly beginning to happen. Much of ML is a verifiable task that can be scaled, and a lot transfers from general coding as well. Leading researchers are predicting their own obsolescence as early as next year, and that is something I take seriously, though I cannot judge the likelihood myself.
As improvement accelerates, we may see generalization happen at a scale that means we don't need RLVR for every profession. We've been surprised by it before.
Is it not, or are the feedback loops just longer? I could imagine being able to build a pretty good simulator of business operation for various kinds of businesses. (Some decent ones already exist as games.) From there, if you can build a good enough simulation, I can't see why you couldn't RL it.
Of course it's easier said than done to generalize it to the real world, where the possibility space is large. But I don't necessarily see an impediment to getting there eventually.
That’s what I am responding to.
Just the other day a model helped me root an old android tablet. Will that still work in 5 years? I don't really have great confidence.
The tech industry cultivated that reputation though. People were just acting rationally by being more skeptical of even larger claims than they were making before.
What's happening today is that anyone and everyone can have access to superintelligence. Anyone with access to an LLM effectively is, simultaneously, a world-class diagnostician, engineer, architect, musician, mathematician... you get the point. This will necessarily lead to a post-scarcity society, since everyone knows everything and will be capable of producing anything.
I wish I shared this optimism, unfortunately that upper echelon of society is the one which currently controls the superintelligence. Everyone may have access to it, but they own it.
Still, as of now, while I use AI for everything I do (mostly Claude but also ChatGPT), I don't use any open weight models (probably more of a mental block at this point), but I do see them as a necessary counterweight to this potential power imbalance.
However, since there is an ongoing war on general computing it wouldn't surprise me if instead people get locked into subscriptions.
I still think of the economic benefit I got from that car. It drove me to university classes. It drove me to work. It took me to see a total solar eclipse. It brought me to see distant family regularly. I went on a lot of road trips on it. For roughly $3k of capital costs.
I look at most expensive things and size them up as "how many Accords is this?"
Almost 7 Accords is a lot.
A post-scarcity society requires MATERIAL abundance in the hands of everyone.
All the material is controlled by ~100 mentally ill people who would rather kill us all than share.
The world of violin playing isn't that large though, so the question is merely getting someone to be really really really good but not quite world class close enough, or not really?
A post-scarcity society is not possible without first wiping out society. If everyone has everything then everyone has nothing. And of course, natural resources are actually limited, and those power LLMs.
It’s hard to take you AI-hype men seriously when you are this ravingly delusional.
Well, the amount of money and resources it took to get here is a reason. I don't think the resources exist for another order of magnitude increase, given the strain on our economy and society we're already under due to AI.
[0] https://epoch.ai/publications/the-plunging-price-of-thought
Reality is extremely chaotic.