upvote
"3. hope scaling stops working before we create an AI formidable enough to pose an existential risk"

I have my doubts. The current AI models are already powerful enough to do some real damage. I am always horrified when I read about people giving Claude direct access to a production system and then being wiped out. My use of AI is usually for the AI to propose something which I then review. But that's not very fast so careless people will usually look better. Until something blows up.

And it's only a matter of time until AI even with the current capabilities is being deployed into military or other critical systems.

I think this will go down like any other technology. We'll ignore issues until there is a real problem. And then hopefully we will do something. Seems with climate change we will soon reach a point where something needs to be done after knowing about consequences already for decades.

We probably also need some massive AI blow ups to (only maybe) do something about it.

reply
Maybe I'm being pessimistic, but we might find ourselves in such a situation that the only practical solution would be to use agents to counter rogue agents. This won't be without collateral damage, though.
reply
Your view sounds more optimistic than mine, honestly. I expect that we will fail to make any serious, coordinated attempt to solve this problem. Then we'll either live or die due to fundamental principles that we currently have no insight into.
reply
If fighting AI with AI is pessimistic, I don't know what optimism is. That sounds like optimistic to me...!
reply