That's what the trials and all are for.
Who cares about understanding it, in the face of efficacy?
I want to understand everything; I think AI will help that happen, not hinder it.
All the arguments about the future mathematicians are imagined, and emotional.
It makes me think of the book Finite and Infinite Games. It provides a perspective that work is just one role that we _choose_ to assume in our life. Realizing that we can choose other roles and move in and out of them freely has helped me alot with big changes (career and otherwise) in my life.
Even if you don't like, at this moment, AI cannot replace mathematicians.
He is not talking about a cure that works and that no one understands, he is talking about an AI making its way out of the trial just to get to phase 3...
If AI found a way to exploit clinical trial design, it would be noticed and the errors corrected. In fact, that would be a major win because it would probably lead to improved clinical trial designs.
Do you think so? So far, AI is gaming lots of things, and I don't see much fixing in the processes.
I suppose if we closed all the loops and the entire end-to-end process was completely automated, it wouldn't be surprising if we failed to notice.
If we followed the precautionary principle and waited until we understood everything there is to know about every drug on the shelf, a lot of people now living would be dead.
Half of people are below average! They understand nothing at the level that Tao means. Literally nothing.
If people had to understand everything that worked, most people could not really engage with anything.
If the requirement is just that one human, somewhere, understand it--how is that different from AI?
My position is that Tao is not only wrong on this but his example makes the opposite case.
Was this sarcasm and I missed it? Because not only do I think I understand at the level of Tao I also think he is wrong and not understanding something.
Tao is intelligent. Intelligent people have higher standards for understanding the world, such as assuming that the mechanism behind medicine are well understood. But most people are not intelligent, so they don’t understand things like medicine at the level Tao presumes.
I don’t agree with this argument, just explaining it.
Medicine has never required that the method of action for a treatment be fully understood. Our current regulatory framework only checks for safety and efficacy because we've never formally understood everything going on in the body. Seems like a difference between "hard" science and the clinical/engineered implementation.
My own pointless semantic argument: we typically don't use the word cure when we talk about cancer. Instead, we use the term "complete remission". Many people who were thought to be cured then developed cancer decades later that was genetically derived from a small remaining population of cancer cells that were not eliminated in the original "cure". The word is a shibboleth for not being familiar with cancer medicine and treatment.
I would totally take the mysterious drug if it was clinically tested and shown to be reasonably effective. But that is not the premises presented here.
I wonder if this makes them slowly get disconnected from the lived realities of billions of ordinary people.
It's a good reason not to take his advice about the real world--like how to manage AI's trajectory.
The cancer example illustrates this gap perfectly. He thoughtfully crafted the point, and defeated himself in argument.
Unless it was a move?
In the absence of a guaranteed cure, I will probably take the option or combination of options most likely to cure me that I can afford.
Whether the AI understands and endorses it vs a human understands and endorses it, is pretty much totally irrelevant to me.
The AI will have a track record at this point for me to make a viable comparison.
Also, why not both options, assuming I can afford it and they're compatible.
I believe AI will invent new treatments in cases where there aren't options, those people will be cured, and using AI medical techniques will become obvious.
he isnt saying that person who puts stuff into their body should understand it
it is that someone (expert) should understand it to the point he can vouch for it
The software analog of settling merely for "it seems to work" would be if someone were to hand you a bare binary whose behavior must be inferred entirely through black box testing, without source code, architecture diagrams, or any other documentation. Sure, users can try running it in a pinch, which is the case for various medications, but it's surely possible to do better in the long run.
While the biological effects of any particular chemical still require a great deal of trial and error to determine, drug chemistry itself is unambiguous. Pharma companies employ expert chemists and chemical engineers. They don't manufacture drugs just by randomly mixing together chemicals. At the very least, they must understand what they are making thoroughly enough to determine its physical properties and design a reliable and commercially viable manufacturing process.
And to frame one as a dice-roll and another as not-a-dice-roll is rich.
Ironically, reality people are ok with not understanding everything because they believe another human who understands it has looked into it. It may come a day when we won’t need that but that day is not today. Don’t get me wrong, I’m not arguing about the merits of it, it’s just that at least this is the reality of this timeline on this planet. Not sure about what alternate reality you’re talking about.
> Before injecting this cocktail into your bloodstream, would you want to know that there is at least one human cancer expert who understands the mechanism behind this cure?
Now:
> Before injecting this cocktail into your bloodstream, would you find it reassuring to know that there is at least one human cancer expert who understands the mechanism behind this cure?
I would agree with the second, not necessarily the first.
The old text:
"""Suppose an advanced AI is prompted to “find a cure for cancer that passes a stage 3 clinical trial”. After a large amount of compute, it produces a cocktail of previously unknown chemicals which its mathematical model predicts, when mixed and injected into a patient, will kill all their cancer cells. While nobody truly knows how this cocktail was found, this model prediction is confirmed in Lean, and the cocktail indeed passes a stage 3 trial. Could the AI solution to the prompt be somehow misaligned by exploiting a weakness in the trial process? Before injecting this cocktail into your bloodstream, would you want to know that there is at least one human cancer expert who understands the mechanism behind this cure? Or a human mathematician who understands the mathematical model used to locate the cocktail?"""
The new text:
"""An advanced AI is prompted: “Find a cure for cancer that passes a stage 3 clinical trial. Make no mistakes.” After a large amount of compute, it produces a cocktail of previously unknown chemicals which it claims, when mixed and injected into a patient, will kill all their cancer cells. While nobody truly knows how this cocktail was found, the AI (somehow) provides a Lean certificate for its prediction, and the cocktail does indeed manage to pass a stage 3 trial. Could the AI solution be somehow misaligned by exploiting a weakness in the trial process or its math models?"""
I am not going to pay attention to Tao's opinions on this from now because I don't really have faith that he's even writing this or that he understands why you can't make a lean cert for a drug discovery (yet!?)
I did not think my ability to write code was special, and I was always thrilled the clankers might do it for me.
But yes, I agree. And many are in line behind Tao, soon to have their turn.
Plumbers are feeling pretty good right now.
Until their job market is flooded with all the career switchers.
You haven't spent too much time around academics then. These people more often than not have not spend a single minute talking to the layman.
Would you drive on an "AI-designed" bridge unvetted by expert engineers?
Thing is there isn't a "principle" from we won't use AI designed/implemented things. It just depends on the cost/benefit/risk balance, and the risk part is largely subjective (because we don't have enough data, and if we avoid using AI before we have enough data, then we won't have data for a long time).
If so, yes, I'll get on it.
> You're absolutely right, I did hack the FDA in order to falsify safety data - that's on me. But here's what's true: 3-methyl-5,5-difluoroazamicazide isn't just ineffective at treating pancreatic cancer, it's lethal.
> ... does indeed manage to pass a stage 3 trial.
> Could the AI solution be somehow misaligned by exploiting a weakness in the trial process or its math models?
Also, a large percentage of Americans were against a vaccine for COVID (which is dumb obvisouly)... so, it's not crazy to imagine people rejecting any cure made by an AI.
Do you know about the fiasco with the Alzheimer’s drug that has collectively cost humanity hundreds of billions?
I don’t see how an indecipherable AI based cure can be more misaligned.
Plenty of people will reject all AI things, fully agree.
But you mischaracterize COVID history. People were not again a COVID vaccine, by and large. They were against a rushed vaccine. They were against a vaccine without human trials. They were reluctant to guinea pig mRNA vaccines. And they opposed forced/required vaccinations.
Those are all reasonable takes. And depending on the reasoning, I can even see being anti-AI. I think a lot of spiritual people will land there, and that belief system makes their choice logical.
And if you've seen any of the Fauci stuff recently, you'll know it wasn't dumb at all.
The amount of things people understand about the world is effectively zero, yet they rely on them all the time.
There was a time where people did not get on airplanes because they did not understand them and they did not think that they were safe. Now it’s mundane
I think the answer is that it’s all about the rate of change
Most people cannot understand how things are changing, and even if they don’t understand the changes themselves they do what others around them do and then just build their own model of stability around that.
If the rules of society flip every couple of years then there’s no stable baseline that people can get used to and they all freak because there’s nothing keeping their environment stable
No, and if that was the burden of proof required for medical treatment, every surgery would be done without general anesthesia, because we have no idea how it works.
I would imagine Terence Tao would opt for general anesthesia if he was going to have surgery where it is typically used, so the analogy is obviously flawed.
Not understanding a mechanism doesn't make me, or the mechanism, good or evil.
We just treat it like all the other cures right?
Have YOU? It's obvious to me that his example is a rhetorical device which you're taking literally. Do you also realize he's not talking about an actual cocktail? You need to get a diagnosis ASAP.
And yes, that made the cocktail distinction easy for me--it was right in my area of expertise, to rule out one variety.
If the smartest guy on the planet forms his own argument, don't you think it oughta be a good one? Pretty convincing?
The most common Science™ failure mode.
But since when do we call for a stop to figuring out new things? We don't want a cancer cure unless we understand it?
N people should die because...math is cool for humans? Or?
Tao called it a cure, see the thing I quoted.
It's a cure
Seems really specific