The maths result is cool on one hand (discovering truths of the universe faster), but on the other there are so many bad outcomes that seem likely, from power concentration to loss of control.
I think AI - like all changes - will lead to some bad things. The internet did too!
But I don't think AI will kill us all.
Interestingly I'd note that the two outcomes you listed (power concentration and loss of control) are dimensionally opposites!
For me this just shows that the future contains such a vast array of possible outcomes that focus on the negatives completely missed the positive outcomes that future also holds.
We focus on stopping bad things because people and systems that don't prevent bad things tend to stop existing. A million good things can happen yet be rendered permanently in vain if one bad terrible thing occurs.
I think we should stop the bad things.
We don't stop building cars because there are crashes, we build better safety systems.
I'm against the doomer narrative ("AI will kill us all") not against a clear eyed approach to making safe systems.