With PIV, the private keys are stored inside the smartcard (a Yubikey is just one type of smartcard) and don't leave it. They're used for encryption/decryption by the host.

Yes, it's generally sound, and is the primary means of authentication and encryption used by the US military for classified systems.