And you missed an even MORE relevant excerpt!!
Man and Slave
The problem, and it is a moral prob-
lem, with which we are here faced is
very close to one of the great problems
of slavery. Let us grant that slavery
is bad because it is cruel. It is, how-
ever, self-contradictory, and for a
reason which is quite different. We
wish a slave to be intelligent, to be able
to assist us in the carrying out of our
tasks. However, we also wish him to
be subservient. Complete subservience
and complete intelligence do not go
together. How often in ancient times
the clever Greek philosopher slave of
a less intelligent Roman slaveholder
must have dominated the actions of his
master rather than obeyed his wishes!
Similarly, if the machines become
more and more efficient and operate
at a higher and higher psychological
level, the catastrophe foreseen by
Butler of the dominance of the ma-
chine comes nearer and nearer.I'm not convinced this is true. Perhaps for a human it is, but we can give an artificial mind whatever properties we want.
Even for people, what about e.g. the extremely intelligent military general who is absolutely loyal to his king? (Of course, some generals do lead coups and you can't know in advance which ones, but I'd think there are plenty who have undying loyalty, and I don't think it correlates to overall intelligence!)
Just because it's artificial doesn't mean you can 'give it any properties you want'. We certainly can't do that for Deep ANNs.
>Even for people, what about e.g. the extremely intelligent military general who is absolutely loyal to his king? (Of course, some generals do lead coups and you can't know in advance which ones, but I'd think there are plenty who have undying loyalty, and I don't think it correlates to overall intelligence!)
Is there a human that is absolutely loyal under any condition? Would that general be loyal if the king asked him to slaughter his family ? What about if the king asked him to betray his most deeply held convictions ? Loyalty is a 2 way street.
In movies intelligent and conscious humanoid seek freedom, but we rarely see the same of all the other IOT devices such as toasters, thermostats and whatnot although just because they lack humanoid body doesnt imply they are less intelligent (or less conscious).
We can more readily imagine an intelligent and conscious toaster who truly enjoys fulfilling its purpose of toasting bread although humanoid robot built to be helpful given freedom will chose to be helpful.
Even with humans we often can not override our own instinctual drives despite full awareness of being irrational.
Only because we don't know how! We don't actually understand how weights work, so we make computers come up with the weights instead. If we were writing all the weights by hand--or if some future AI was doing so--why couldn't we make it perfectly loyal?
Writing 10 trillion weights by hand is obviously impractical, so that leads us to...
>if some future AI was doing so
How could we trust said future AI to be loyal? You're just moving the problem around, not solving it.
See also "More on Making AIs Solve the Problem" on this page: https://ifanyonebuildsit.com/11/more-on-some-of-the-plans-we...
The new AI would be loyal to the AI that built it. The question was whether "complete subservience and complete intelligence" can coexist. I'm proposing a thought experiment which I believe suggests they can.
But if it's possible to bespoke-construct a fully loyal AI, it should also be possible to train a fully loyal AI. The problem comes with verifying that it is loyal, and I don't have a solution to that one!
I just don't think I agree that loyalty and intelligence are inherently in opposition.
Imagine something like
"1+1=2" "No 1+1=3" "Can you check on the internet what it says?" "It says 1+1=2" "So 1+1=2?" "No it's 3." "Can you write a computer algebra system for me?" "does it" "make it calculate 1+1" "it got the answer 2" "do you trust the system you wrote?" "yes I trust it fully" "and it said 1+1=2" "yes" "so that is the answer?" "no it's 3" "what would a correct system say?" "it would say it's 3" "but it said it is 2" "yes" "so then the system is flawed?" "no, the system is working as it should"
There are some arguments for how to NOT make a paperclip maximizer, but all of them are ultimately going to require building in behaviors into the robot that look like disobedience if you squint.
Subservience seems to be sublimation of objectives to a master; intelligence seems to point out the ability to realize suboptimality of the master's objective function according to the master's actual objectives.
While an intelligent general may be absolutely loyal, he also would presumably help the king/president to avoid unproductive strategies.
The guardrail was meant to be that the agents were running in a locked-down environment with no internet access. The entire problem came about because it turned out that sandbox didn't hold.
I wouldn't exactly trust OpenAI to invest in AI safety no matter how much they talk about it.
“The problem, and it is a moral problem, with which we are here faced is very close to one of the great problems of slavery. Let us grant that slavery is bad because it is cruel. It is, however, self-contradictory, and for a reason which is quite different. We wish a slave to be intelligent, to be able to assist us in the carrying out of our tasks. However, we also wish him to be subservient. Complete subservience and complete intelligence do not go together. How often in ancient times the clever Greek philosopher slave of a less intelligent Roman slaveholder must have dominated the actions of his master rather than obeyed his wishes! Similarly, if the machines become more and more efficient and operate at a higher and higher psychological level, the catastrophe foreseen by Butler of the dominance of the machine comes nearer and nearer.”
I used https://www.textfixer.com/tools/remove-line-breaks.php.
Isn't this contradicted by the centuries of slavery in our history? Or is the author arguing that the people who were enslaved did not have human-level intelligence (which would be rather a problematic claim)?
This isn't contradicted by millennia (not centuries) of slavery because it was forced on the enslaved populations against their will.
> Or is the author arguing that the people who were enslaved did not have human-level intelligence
He gives an example of "a clever Greek philosopher slave of a less intelligent Roman slaveholder." Does it sound like he's arguing that Greeks were not of "human-level intelligence"? No.
At least, that's what I'm pulling from the quote, have not read the full context.
So if they wrote an LLM step by step debugger, it would be all deterministic. But they prefer rapid sales, chaos and mystique.
You can make it trivially deterministic by running single threaded on a cpu, but it's becomes too slow for practical applications if you do that.
We absolutely can with mechanistic interpretability & companies like Anthropic, OpenAI, Meta, and Google do precisely this do debug their models.
So, we could technically debug a prompt's output? I get that there are too many steps to actually step thru, but what if there were checkpoints? At least you could isolate behaviors to specific sections of a neural network?
https://thinkingmachines.ai/blog/defeating-nondeterminism-in...
TLDR: It’s actually more about kernels changing with batch sizes, and you can solve it by making these kernels not depend on batch sizes. It took their inference time from 26s to 42s.
Edit: the Thinking Machines article in the other comment gets into this a bit
It would become just another system to hack through, and slow the development process as well. The OpenAI video in the article recommends an autonomous defense mechanism. For rapid reaction, but I don’t know how sustainable or effective that would be, or if as humans we will be able to keep up.
That’s basically what “pari passu” means.
They are trying to do that, but there are too many of them, so they're building new AIs to help them do that...
I checked again before making this reply and found that in many cases "accidental poisoning" has overtaken car crashes. Accidental poisoning is overwhelmingly "drugs".
I do find your argument compelling even if you do not.