Around the same time Citizens United was decided, we also got McCutcheon v. FEC, which invalidated campaign contribution limits basically completely. If we take the logic of Citizens United at its word - that money is speech - then letting someone drop billions of dollars to change an election is like firing a sonic weapon at a bunch of protesters to silence them. So, right off the bat, we have a situation where protecting the "speech" of the rich and powerful directly imperils the speech of everyone else.
But it gets worse. Because we got rid of campaign financing limitations, there has been an arms race with campaign funding that has made all speech completely, 100% pay-to-play. We have libre speech, but not gratis speech.
This isn't even a problem limited to merely political speech. Every large forum by which speech occurs expects you to buy advertising on their own platform now before you are heard. If you, say, sell a book on Amazon or post a video on TikTok, you're expected to buy ads for it on Amazon or TikTok. You are otherwise shut out of the system because discovery algorithms want you keep you in your own bubble and you're competing with lots and lots of spam.
You think you are reducing the influence of the rich, but you are actually just raising the price of entry. A millionaire can donate to a PAC and buy TV ads, but a billionaire can buy or start a newspaper, TV station, or social media network. What are you going to do then, tell the newspapers what they are allowed to print?