Well, that we have to put an insane amount of work to keep it aligned. Kind of like pushing a huge round boulder to the top of mount Everest. You have to expend energy to get it there and fight physics to keep it there.

With a static model we might be able to keep it somewhat under control, but think about future continuous learning models. They'd drift away from unstable high energy configurations. Also any model being trained by people that don't care about safety.