Maybe it's the same probability of being a problem, but it would be a problem introduced by different kinds of threat.

If you're worried that you'll be targeted as an individual by somebody who cannot coerce a CA but can tamper with the channel you're using to get the public key, use https.

If you're worried that you'll be targeted along with many others by somebody who can coerce a CA, use ssh and ensure that some folks are comparing key fingerprints such that at least one is close to the source.