upvote
The "global stop order" is just generally perceived as an impossible coordination problem. So instead we see a mix of labs voluntarily putting in guardrails and regulatory efforts (which are not only aimed at hypothetical super-AIs of the future). Of course labs are also in a competitive race. And I actually think that it does make sense that the richest companies in the most dominant positions would in a better position to worry about safety than a startup that is just trying to survive at all. And just in general, it seems reasonable that the fewer companies have access to dangerous tech the better. This isn't really about some highly speculative future tech either -- current models already pose lots of risks, and the pace of model improvement is something wildly unprecedented. Whether or not you call it ASI, the capabilities we will have two years from now are hard to even imagine properly. Also, I don't think the issues that you are highlighting are all ones that Anthropic would dismiss as second-tier. In particular, mass unemployment from AI is how we will deal with a massive devaluation of human labor is one of the most serious concerns. And about other issues, reasonable people may differ. I'm more worried about biorisk than environmental damage, for example, but clearly we should be keeping an eye on both. Serious risks and problems, just because they aren't already harming people today, are not just a distraction.
reply