Why not ?
Even if you don't want to go all the way to SSH certificates, SSH public-key authentication is laughably easy to do.
I also don't buy the argument "but its on a ULA address". Attacker's old favourite these days is finding a way in (North–South) and then using that as a stepping stone (East–West). Also the general concept of "lower security is OK because its on the LAN" is so 1997.
No excuse for doing that these days.
You can use a Yubikey or you can use your computer's secure enclave or TPM with FIDO at no extra cost.
A password is never, ever "strictly better". Seriously, you're just making shit up now.