upvote
This is how most people already live.

The average person doesn't know how the medication they take works, the mechanics of climate and climate change, how the energy they consume is generated, etc.

reply
In the case of medicine it goes deeper than that, sometimes nobody really understands why a medication works.
reply
> nobody really understands why a medication works.

But they work hard to try to understand it because the more they do the better the results for peoplea health. Also it goes without saying that if someone didn't understand many of the things that we do understand then things would be worse for us all.

reply
Sure but in this reality, humans won't need to work hard to try to understand it for there to be better results for peoples health.
reply
I feel like we just don't understand a lot about the human body still. Only a century and a half ago we had a US president die because a doctor rummaged around and damaged internal organs (with unwashed hands, because germ theory was still not uniformly accepted) to try to find a bullet on the wrong side of his body. We've come a long way since then, but it's not like we've been doing the field of medicine rigorously for all that long. The best strategy we have seems like pretty much the same as any other scientific field; come up with ideas based on what we know, try them out, and see if they work. That doesn't necessarily give us any actual insight into why it works though.
reply
Yet it takes rigorous years long tests to understand effects of a medication.
reply
Yes, and understanding the effects of a medication is different to understanding how the medication has that effect.

The medical field as a whole isn’t generally interested in understanding how medication, only in empirical measuring and qualify the effects.

reply
> The medical field as a whole isn’t generally interested in understanding how medication, only in empirical measuring and qualify the effects.

I don't think this is quite correct. I mean many practitioners of medicine will have the attitude of ... "if it works, it works". And that's perfectly reasonable.

But if you understand the mechanism of action of a drug (or other treatment), it (often) makes it easier to improve a drug.

So ... some sectors of the "medical field" understandably care only about empirical results. But other sectors would prefer to understand what's going on.

reply
>The medical field as a whole isn’t generally interested in understanding how medication, only in empirical measuring and qualify the effects.

This is nonsense. Most, if not all professionals are interested in mechanism of action, but without Ms Frizzle, it is extremely difficult and expensive (time and money wise) to figure that out. So while the labs run the experiments with the very limited funding they have, we make do with using the second best thing we have, which are statistics.

reply
The tests are there to confirm with statistics that (1) it works and (2) side effects are not too bad or too frequent.

That doesn't mean we understand why the medication works.

reply
Yes and no: understanding means to know the relationships to the underlying system.

When you already know that system well, those effects are often just a matter of simple inference.

Just like here: most people are actually perfectly capable to foresee the detrimental effects of abandoning understanding.

Living in a fantasy world of "magic" makes you dependent upon your caretakers, who provide the ingredients.

reply
And this is exactly what Dario wants.
reply
unless its the mrna covid vaccine. which needed how many months?
reply
This is so tiring.
reply
That cant be true, theres a paper trail of sign offs of the most likely people to understand the probabilities.

Please define proof of "really understands why a medication works"

It reminds me of fynemans why do magnets work. Yeah sure does anyone really understand anything? Its metaphors all the way down

reply
We don’t really understand how NSAIDs reduce pain, as a concrete example. In contrast we have a pretty good understanding of the pharmacology of caffeine.

It’s not like magnets, some things really are gaps.

reply
Some medications are designed for one purpose, then other effects are discovered in practice. Gabapentin, for example, was designed as an anti-seizure medication structurally similar to the inhibitory neurotransmitter GABA.

Now it is primarily used to treat neuropathic pain, and the mechanism for that is not well understood. The GABA receptor is not involved. This effect is just a happy accident, and nobody really understands why it works.

reply
its a good thing that our lack of understanding of medicines has never killed anybody or have had any intended consequences

oh wait

reply
This is also how most CEO live.

They ask their employees to build stuff, and have no understanding whatsoever of how any of it works.

reply
A major school of German sociology places this at the core of its theory. According to this theory, modern societies are characterised in particular by the fact that individual subsystems of society reduce the complexity of their own (sub)system to the other systems to enable them to act at all; they fulfil what is called an "Entlastungsfunktion" (relief function). Prominent representatives of this school of thought are Max Weber, Arnold Gehlen and Niklas Luhmann.

In their view, it is modern institutions (public and private) which, as supra-individual entities, have long since become autonomous systems. The fact that the individual office-holders are human beings, meanwhile, is of little significance.

Hannah Arendt, in her theory of totalitarianism, attributed the effectiveness of both Nazi and Stalinist policies of extermination to the largly moral indifference of bureaucracy as a system.

In this sense, the task of controlling AI is a variation on the problem of harnessing a complex society consisting mainly of autonomous subsystems. This is a problem which has increasingly challenged humanity already for quite a long time. And it has been very difficult so far, even without AI ...

reply
That's what makes libertarian AI doomers so strange. "Letting unconstrained machines with human components loose on the world will be okay, but letting unconstrained machines with nonhuman components loose on the world will produce terminators." How can it be both one and the other?
reply
I'm going to say it's how everybody already lives, because while somebody may know some of those things there is nobody who knows them all.
reply
But we can generally assume that the people who built and understand some technology will have the same desires and worries as us, and are bound by the same laws. For instance, if it was a team of strangers who designed the terawatt fusion plant, we can still assume with some confidence that they care about safe plant shutdown, failure recovery, ensuring there are no toxic side effects of operation, etc. Can we say the same when it is machines doing everything?
reply
Very interesting point. I think the counter-argument to that is that the complex modern society is based on somewhat “deterministic” systems, in that, even if a single decision or event isn’t rationally explainable in the moment, at least in the aftermath, it typically becomes understandable, maybe even reproducible. There is someone, somewhere, capable of explaining, maybe even multiple someones.

We don’t generally have that insurance with LLMs/AI, yet?

reply
Imo the counter-argument is that in principle you can research and understand the mechanics of climate change or medicine. You just don't have the free time and/or the motivation. With AI that changes - you could spend your entire life trying in vain to understand its reasoning.

It reminds me of how some religious people say that science is effectively no different to religion because we all take expert opinions on faith. But the difference is that there is a well-defined pathway to understanding, if you wish to do so.

reply
This seems to me like asserting everything in the universe is explicable by physics. It may be technically true, but still not relevant to understanding earthquakes.

(Don’t bother to argue this not true unless you disagree with the essence of the argument.)

Anyway, post-hoc explicability isn’t a counter-argument to the assertion that almost everyone takes almost all technology as magic, from medicine to computers.

I’m still trying to understand your argument. Are you saying that after the fact we understand AlphaGo move 37? But somehow we are never going to understand an LLM’s decision afterwards? Seems like a disconnected take to me.

reply
There are huge systems supporting that ignorance though. And where those systems are breaking down people do care.
reply
This is a very Western take though.

The problem with "ignorance" in Western countries (particularly the US right now) is that it's very common for people who don't know to believe they know and form ignorant opinions that they often want to be applied society-wide in some way. You can see this with everything from climate change to vaccines.

In much of the world, even in middle income countries, people are comparatively poor and, in my experience living abroad in such countries for many years, much less concerned with "understanding" and forming opinions about everything under the sun. It doesn't mean they don't value education and are opposed to development/progress, but it does mean that they don't question whether the vaccine they're taking is the product of a conspiracy, think too deeply about why the river is flooding more often, etc.

They just deal with life the best they can and are more focused on supporting their families, enjoying what they can, etc.

Culture and religion play into this. The way secular and Judeo-Christian people look at the world is very different than, say, Buddhists, Muslims, Fulani tribespeople, and so on.

reply
the average person doesn’t have an internal monologue

can’t use a computer (they’ve had like 30 years now in first world developed countries)

many can’t even use their smart phone beyond calling, texting (many can’t type well), and doom scrolling (they get addicted to drugs, gambling, and other LCD activities)

many read at a 6th grade level. most can’t even calculate tip in their head.

meanwhile, the same smartphone can give them access to literally any information and knowledge the world in seconds. and now gemini can explain stuff since most ppl can barely read or think.

it’s sad out there.

but more importantly. it’s not my problem.

reply
No single raindrop believes it is to be blamed for the flood
reply
> but more importantly. it’s not my problem.

But it absolutely is! Those people can and do vote.

reply
> But nobody can understand anymore how any of it works. We just ask and then trust the AI to deliver (as it always has).

> Isn't it fun to imagine how life would look like in that scenario?

This is horrifying to me.

reply
Rightfully so.

People succumb to defeatism and acquiesce to regressing to zoo animals, with AI as their caretakers.

They simply cannot help but to apply the economic gauge of short term profits to value the alternatives.

Even though, obviously, here long term human survival and living conditions are at stake, necessitating an entirely different set of considerations.

reply
Some of it has to do with the fact that many people pushed forward training themselves in a way they understood would better themselves for the future. They put in time, and now their sense of value is in question. They're forced to think about things like value (of life, themselves, and hopefully others), and the things they want to remain valuable in the future.

It's interesting to me that you only mentioned the people using 'AI' in the 'short-term' ways, and not the ones that use it to better themselves in the 'long-term' ways. You can spend your own time focusing on either usage, it's really up to you and your concerns. Either group's sense of value is what determines their behavior. Where they spend their time and thinking must be elsewhere, and you disagree with it. Who judges the quality of time spent? You do.

Is it more useful to think about self-improvement, and how to navigate the future in ways that might help you re-establish value of yourself, life, and others? Acquiring knowledge is a struggle, the author mentioned this. There is also Plato's Allegory of the Cave, which highlights some of that struggle, a resistance to change. And we're all limited by time, our genes, our station in life.

The only way to help anyone out of the cave, is to help them believe something different about themselves. To help them believe there is good reason to spend time going deeper into knowledge, or at the very least, allow others with the passion and station for it to do so.

If not for the very least reason that it keeps us 'in the loop' of some central idea behind intelligence (prediction?). Or because we feel it keeps us safer, as a fallback measure because we acknowledge we have to trust other's knowledge to exist.

reply
AI is literally owned by james bond type villains.
reply
> Isn't it fun to imagine how life would look like in that scenario?

I think there's a lot of sci-fi out there that already did. Maybe it's not utopian because a pure utopia would not be likely to have an interesting story, but on the other hand, most huge technological advancements end up having just as much potential to reinforce existing power imbalances in society rather than solve them. It's not obvious to me that if we got magic super AI that can solve every scientific problem in society that it gets used in pretty much the same way as anything else: making the people who control it a lot of money rather than sharing the power with everyone without charging them.

reply
Here's the thing: nobody is stopping you from putting in the time to understand all that. The problem is, nobody has that much time, and we get hungry. And so we want the community to move with us, spend the time the same way as we do, to ensure value. We are all saying: we want someone else, others, to put in that time for us. The truth is, we all want quality, and value is closely related.

I can almost guarantee you the first time you show cancer symptoms, you won't care whether the cure came from an AI or human's understanding. But we haven't seen that, so we can't make the judgement call.

reply
[dead]
reply
We would care about having fun.

For some people, fun is doing physics and mathematics. So they are going to keep doing that.

reply
> We would care about having fun.

For even more people fun is TikTok, Snap, Instagram -> sounds like a collapse of a civilization to me if you increase the ratio even more towards dancing kids sharing their content non-stop with no added value to the society

reply
It's not fun, it's a physiological addiction, because we learned enough about our brains to hack dopamine and reward cycles for profit. It's literal abuse.
reply
yeah, a lot of people are just watching 1 video after another.
reply
Before extrapolating that far, take a look at the frontier labs' own job boards (https://openai.com/careers/search/?). Isn't it curious that they are still recruiting human "Android Engineers", "Account Associates", "Consumer Marketing Leads" instead of automating them with their world-beating models?
reply
Yes today but maybe no tomorrow.
reply
Ian M Banks covers this in great detail in his science fiction books. Highly recommended.
reply
Yeah, the Culture is what happens when this transition goes perfectly right.
reply
You’re describing a world where no human being has any agency. Curing cancer etc. sounds great but we’d be losing something priceless in the exchange
reply
Our best philosophers have already pondered this question, and showed us the answer in the form of humans on the Axiom starship in WALL-E.
reply
> We would probably no longer care about code, engineering or even physics and mathematics among other things

This sounds like a boring existence. I take your meaning, but want to point out that not everyone learns about things because of practical utility, some of us find it incredibly satisfying to learn how things work just for the sake of learning.

reply
Sure but no one's stopping you? I know a great many things of no direct practical value to me.

There's also things I don't know and don't have the time to learn which are very helpful to have AI do for me: web interfaces are really useful and I look forward to them now working exactly how I want. I'm not ever going to regret not spending more time trying to figure out how to center divs or which framework I should use because they're all deprecated.

reply
Well there is this one framework, CSS, that is quite alive and well :)

But point taken.

reply
You can't just leave us hanging like that!
reply
It's about as fun, and as realistic, as imagining magical ponies and people with superpowers?

This is such an incredibly naive and absurd vision; we've already proven that humans are very often very bad at implementing other humans' good ideas. There's nothing that AI is likely to bring that will improve this discernment.

reply
I’m reminded of themes from the Hyperion Cantos! Maybe my mind is over-connecting, but it’s not the first time I’ve drawn similarities in the last few years.

It’s terrifying to me to think we’d let AI make things for us we never understand. Like livestock not knowing how auto-feeders dispense their daily food were built and appeared, they just gladly eat until…

reply
But where are the “cure for cancer” papers for it to ingest and then give to us?
reply
I think the weirdest part of that scenario is that science might become something closer to archaeology
reply
The term you are looking for is hermeneutics.
reply
Love it! I am now a Software Hermanuticist
reply
Hmmmm. Yeah plausible.
reply
> Isn't it fun to imagine how life would look like in that scenario?

Look at the financially desolate subcultures with no option for advancement or dignified life.

That is the goal and that is how it will lool like, if the tech CEO managed to gain the power they want.

reply
Those subcultures are like that because they are small. If that became the lot of most people, society would look very different. The choice would be between Fully Automated Luxury Communism and Oligarchic Hellscape.
reply
> We would probably no longer care about code, engineering or even physics and mathematics among other things. We would probably mainly care about

… about what?

reply
There is no answer. That's why this is such a terrifying moment for humanity.
reply
Frying our dopamine receptors as fast as possible.
reply
deleted
reply
no
reply
STEM will be considered "historical studies" in an AI-ruled future.
reply