> Well, that didn't happen.
Predictions are not false because their claims are not in the past yet. That bar would make all predictions wrong.
The faster things happen, the higher the speed-of-progress expectation bar gets raised. This is how objectively compounding progress gets interpreted subjectively as linear, or even as a stall. Despite the dramatically increased rate of progress compared to the context of decades or centuries of speculations about cognitive machines.
Models being used to write a lot of the code for new models is a strong suggestion of compounding capability. With new models achieving higher scores. Not proof, but a high bar for evidence that we may be in that explosion now.
The fact that models trained to match SOTA model behavior (i.e. distillation), now learn much faster and more cheaply, than models trained on human behavior, is also strong evidence that capabilities are compounding.
I agree, that's why I said "we're seeing things we have working towards this". I think that the "jagged intelligence" that is often used to describe our curent models is confusing a lot of people. On the one hand you have models failing basic "trick" questions that a 5yo would get, but on the other hand you also have models that write better/faster kernels that can run faster/better models, and are currently used to improve the next generation (via dataset prep, filtering, RL environment creation/curation and so on).
To me it's an interesting lesson in how hard it is to predict technology advances which still applies to our own predictions of the future.
>> Well, that didn't happen.
> Predictions are not false because their claims are not in the past yet.
"Within the twentieth century" places it definitely in the past, and definitely false.
Also, of the early AI writers, I think he was the least doom-y. Norbert wiener and I think von neumann were very concerned about the social effects of the singularity (von neumann's word for it).
Vernor Vinge has an 80s essay calling down to Good a lot, and credits him with a 'meta golden rule' which aspirationally says that what comes after us will care about their origins.
It’s not uncommon to find these kinds of predictions from the twentieth century, which of course with hindsight can seem wildly optimistic.
But I think there are understandable reasons for that optimism.
Imagine you were born in the early twentieth century, say roughly 1910 +/- 10 years. If you had been born then, you would probably have lived through all of the following:
- The transition from horses to motor vehicles - The invention of the airplane, from first flight to supersonic - Not one but two world wars - The splitting of the atom - Human spaceflight, including traveling to the moon
If you’d been born slightly earlier, you could also add electrification to that list.
When I compare what I’ve seen in my life so far, I don’t think anything comes close (for better and worse) to even one of those things. So yeah, of course they were optimistic about technological progress—look at what they just lived through.
> Hahaha, this is straight out of 60s-70s sci-fi, where their best futuristic interfaces were smaller CRT screens / flashy keys, etc.
Actually that is not far off from how high bandwidth interconnects work, but through other media.
And... I hate to say it, but I've been interacting with computers connected to WiFi about 90% of the time over the last week. Rarely used a physical Ethernet link.
Technically of course the path is continual evolution, but the applications are wildly different.
> Well, that didn't happen.
Not only that, but in my cynical eyes the proliferation of LLMs has triggered a stupidity explosion. Either that, or it just made it blatantly obvious by how much stupidity we have been surrounded the whole time without realizing it. No other development demonstrated so clearly that the dark ages where we believed in sorcery and miracles have never really ended.
I'm sure it's my fault, somehow - but I'm a damn good writer; I just don't know how to communicate with people who won't put forth the effort to engage with detail.
The latest advice is "put in through [LLM]". Which, yeah, takes out the details, and makes bullet points, but sometimes the details are the point... Right?
Higher-level decision-making, I find, isn't so much like that. Sometimes it's just tradeoffs: "if the company's going this direction, then A is better; but if that, then B". But they haven't even recognized the this and the that - and don't want to think them through transparently - so I've got to choose the A or the B in the dark. Actually, it's worse than that, because some people have (by implication) already committed to the this, and others to the that, but neither side wants to talk about either, because that would create conflict. So, the best (job- / career-wise) choice for me is to predict which of A or B best satisfies whichever side will be most influential when the bill for A or B comes due. Either could be (technically speaking) correct, but the choice is never made on a technical basis.
The military had stealth aircraft in the 70's. I'll bet they had LLM or better in the 80's and the tech we have now is the consumer-grade version they seeded into industry in the 2020's.
For reference, a single 4090 GPU has more FP8 flops than the top supercomputer in 2007. A 4x 4090 computer (something you can buy today for ~10k) would be better than to top supercomputer in 2010 [1]. There's a reason deep learning only started to really work in the past few decades. We had the "theory" for a while, but no compute to actually put it in practice. And the current models are being trained on 10s to 100s of thousand of enterprise GPUs, that make the 4090 look like a toy.
[1] - https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...
NASA used Commodore 64's as part of the constellation of computers needed to launch the space shuttle. That whole program was pretty hokey. That silliness doesn't preclude some radically advanced compute tech from existing elsewhere at the same time. Of course something like that would be ultra secret and you and I wouldn't be told about it.
You have to be told something before you can believe it, right? You also get to choose whether you believe that something you're told is 100% god's honest truth or a load of bullshit as well. It's kindof sad to see people try to gatekeep thought here. Lot of rigid thinkers.
I recall Intel in the 90's trying to build clockless pentiums. They couldn't figure it out or maybe it had integration problems with clockful peripherals and it never came to fruition. But I wondered if that was some furtive attempt to seed some tech into industry. I just try to keep an open mind to these sorts of things.
https://en.wikipedia.org/wiki/FGM-148_Javelin#Seeker
It was featured in one of the Modern Warfare games from around 20 years ago, and I looked it up because back then, I called bs on the fire & forget aspect.
An open mind doesn't mean "a mind full of open circuits."
Why are you so certain you know what tech existed in the 80's?
Useful LLMs could not have been built without deep-UV semiconductor fabrication at scale, which in turn can't exist without a massive, complex supply chain that envelops the globe like a spiderweb.
Such technology could not have been built speculatively, or without people noticing, or with the physics and material science of the day. We didn't even know how to make blue LEDs then!
They didn't have LLMs but they did have "AI" already (for a fair while by then). It wasn't much anything like what we have now really, but it did exist and by the current standards of that time period it was pretty much straight out of science fiction. (Imagine how shocked they'd be seeing what all we have now. "Supercomputers" in nearly every pocket, widespread broadband Internet, LLMs, etc, etc.) You're definitely right though in thinking that they had technologies far beyond what they told the public about. That's been the case since before my own lifetime at least, and absolutely certainly still true today.
Inventing, learning about, and exploring new technology and ideas, just as many experiments we do today are. :)
and got upset when Anthropic didn't want to let them use their LLM
I see those trillion dollar data centers we're building in the US are classed as military installations or at least appear to be a public-private partnerships where the data centers are being built on air force bases.
Perhaps the consumer compute tech has advanced to the point that it's good enough to host the next phase of intelligence for the military. Maybe all the water AI datacenters need is for energy, not cooling.
somewhere around 1990, working at a Dod lab, I picked up on the 'neural network' craze and did a little work. it seemed interesting, but not that much different than using adaptive filters for signal processing. that seemed to be the general consensus, that there wasn't a lot of of interesting behavior or depth there. the number of weights and sizes of super computers at the time were a good 3-4 decimal orders of magnitude lower than what's being thrown today.
so unless the military ran a giant expensive program just to hide the fact that they actually had machines 10000x faster than the $50M (in 80s money) supercomputers of the day, and had trained them on a corpus they whipped up out of nowhere (the amount of information on the internet at the time could probably have fit on a few of todays phones), it probably didn't happen.
Well, did they? They do fritter away a lot of money. Maybe there's a long-running black budget somewhere that could account for it.
You are framing your argument in technologies and technological progressions that you are familiar with. Reality isn't so neat though and there are surely many technological paths to artificial intelligence they could have exploited.