It's completely crazy to use passwords when you needn't. Passwords are a human readable shared secret, they were already obsolete when SSHv1 was invented last century.

From the outset SecSH (SSHv2, the thing you actually use today and if you're younger, likely the only thing you ever have used) has public key authentication as a Mandatory To Implement feature. Implementations where that doesn't work aren't even SSH, they're garbage.