- Put the public authorized keys in /etc/ssh/keys/%u immutable, monitored for checksum changes.
- Have CIDR restrictions in said public keys.
from="x.x.x.x/n,x.x.x.x/n" ssh-ed25519 AAAA[snip] Text-Identifier
Did you create your own process for monitoring/alerting on the checksum?
> This post documents how I’m using various secure enclaves (YubiKey, TPM, Apple Secure Enclave) to hold/encrypt SSH key material and achieve the goal of not having any unprotected key material living on-disk.
Could you help me out and state the goal of your practices as simply as this? I can guess, but I'm really not sure, especially when you get into nftables checks for window sizes.
- One must connect from the correct address and with the correct TCP options.
So for example I could give you one of my ssh private keys and it would be entirely useless unless you were on my home internet connection using a device that emits my TCP options. (Or this could be a corporate SNAT pool or corporate VPN pool.).
Even the most clever MitM (man in the middle) attacks become significantly harder to perform. Leaks happen, I reduce the blast radius of the damage.
Correct. Thankfully none of what I am doing is security through obscurity. It's actual security with a subtle scent of obscurity. If it was just obfuscation and you knew something, then you could do that something from somewhere else which you can not.
1. -sk keys on Yubikeys (with touch enabled) are my primary keys, used for signing and always installed on every host I touch/etc. I initially planned on using the PIV slots on my Yubi's to store the SSH keys instead of the built-in FIDO2 setup, but I hated having to manage multiple moving parts (like getting pscd running on all my clients). The only main benefit of using the PIV slots was being able to set the "cache" touch policy: the YK would cache a touch for up to 15 seconds so doing something like "signing multiple commits during a rebase" would only take one touch instead of num_commits. The side benefit would have been getting to re-use the same SSH key across primary and backup Yubis. The simplicity of the FIDO setup won out and I decided to treat getting annoyed at pressing my Yubikey a bunch of times as a sign I should be doing some sort of alternative setup like a dedicated key. Most prominently, I've now properly setup my Ansible configs with dedicated deploy keys kept as secrets instead of reusing my personal key.
2. I use 1Pass and Bitwarden (work/personal) to store a secondary SSH key that's only used for basic Git auth or other low-risk tasks. This SSH key is loaded from the Password manager directly into ssh-agent via stdin so it never touches my disk; I use fnox to make this backend agnostic.
3. Work uses Teleport, personal uses Tailscale so I rely on those SSH auth most of the time. Nothing much to say here except that Tailscale ssh is really convenient :) .
I've also setup PAM login and sudo on my machines, which brings me back to the old Sun Ray/Smartcard days. None of this is really to prevent security breaches/dedicated attackers or anything, though it certainly makes me harder to compromise. I just wanted to make absolutely sure I couldn't accidentally upload an SSH private key somewhere.
I have a handful of security keys, so I have a helper script to drop them into new hosts and another helper script on my systems to import keys straight to ssh-agent once detected.
Has worked very well and I don't think I'll ever go back to a "password manager as SSH agent" again, but it's probably the best alternative backup. I still have Bitwarden holding some backup keypairs for this reason, but haven't needed to use them in months.
I can't imagine storing my SSH keys on a TPM that might reset them after changing some sticks of RAM changes the hardware measurements or some firmware bug triggers the self-erasure....
If you have just one Yubikey and you lose it, then you lose your SSH access.
But that's true as well if you have just one SSH key on your computer and you lose your computer, right?
I have offline key on two different devices, haven't lost one yet over the last decade!
> All YubiKeys have a storage temperature range between -20°C to 85°C (-4°F to 185°F)
https://docs.yubico.com/hardware/yubikey/datasheet/_static/Y...
Maybe they should work on that, otherwise Yubikeys won't be operating during the summer soon...
1. Require a PIN followed by touch on the first use after inserting the YubiKey
2. For every following authentication while the key remains connected, only require touch
The extra GnuPG setup needed to make this work is unfortunate, but AFAIK this still isn't possible with fido -sk keys on the YubiKey.
Edit: well, maybe not exact. With -sk keys you’d have to type in a passphrase instead of a pin. But that passphrase could be a pin.
1. This method only works (securely) for non-resident keys. 2. hence requiring having pk stub copied manually to every machine 3. its passphrase is prone to brute-force
While piv based ssh keys solve those 3 problems (with one caveat, left for a careful reader :)
Thanks for the tip! I'll try that this evening, I'd love to move to FIDO keys if possible.
`ssh-keygen -t ed25519-sk -O resident -O no-touch-required -O application="ssh:nameyourkey" -f ~/.ssh/id_ed25519-sk`
and
`ssh-keygen -t ed25519-sk -f ~/.ssh/id_ed25519-sk`
?
The latter is what I always do. It creates a file that is useless without my yubikey being present, but is it not actually resident on the key?
In the first one the handle file that's created, can be recovered from the yk key itself, that's what resident is.
Having the only practical difference between the two is that, with the first one, you (or someone else) only needs the yk to use the credential. But in the latter case, one needs the yk and the handle file.
The downside with resident keys is that someone stealing the yubikey can see what resident keys are on it, and then possibly use those keys (if they aren’t protected). Newer firmware comes with ways to protect against this, but you’d have to enable it.
[1] has more details
1: https://developers.yubico.com/SSH/Securing_SSH_with_FIDO2.ht...
Anyway, resident key means that you can extract the public key from the yubikey. Say someone steals your Yubikey Fido2 PIN and your Yubikey, they have everything.
Non-resident key means that the attacker needs to get that "useless" file you describe, too. And if you lose it, then you can't use the key anymore.
Another difference is that a resident key takes space on the Yubikey, so you can't have infinitely many. Whereas you can have infinitely many "useless" files.
I don't understand why the server should enforce this...
They mention twice that Yubikeys are expensive, but for SSH their cheaper FIDO2 keys work:
https://www.yubico.com/product/security-key-series/security-...
I also have a Yubikey but it's been a few years since I tried it for ssh keys. I use it as a regular security key. Maybe I should try it for ssh keys as it sounds like the tooling has improved.
What happens when you replace the ThinkPad?
This means someone with access to your filesystem can’t just copy the key off.
Of course a passphrase is another protective layer. But passphrases can be keylogged, or the decrypted key (or passphrase, depending) in the memory of an ssh agent can be can potentially be read out by an attacker.
With separate hardware, the assumption is that the hardware is much much harder to attack, so the keys stay on there and only on there.
If someone steals your Yubikey without compromising your computer, they cannot use it without the passphrase.
If someone compromises your computer to copy your passphrase, AND steals your Yubikey, then they can use it. But you will know that you are not in possession of your Yubikey anymore, and you can do something about it.
I really don't see how you can blame Yubikey for this one, your employer is entirely to blame for only supplying you with one Yubikey.
Your hardware fault is unfortunate, I've been using Yubikeys for many years without a single issue. But ultimately that is beyond the point here .... come on dude, you work in IT, surely by now you know you don't put all your eggs in one IT basket ?
Not following fundamental best-practice and then complaining when things break is really a "move along, nothing to see here" moment.
While this is true, you can create static hierarchy keys that would not resolve into separate key files for the TPM. The issue here is that anyone with access to your TPM can recreate the keys, unless you set a owner password.
https://man.archlinux.org/man/ssh-tpm-agent.1#Hierarchy_keys
I always worry what do you do when your hardware fails
There is only one answer to your question: It is irrelevant.
Yubikeys are not that expensive, and anybody who depends on Yubikeys should have minimum two of them.
This isn't a Yubikey thing. Its a basic IT equipment redundancy 101 ... if its important, minimum N+1 it.
The same reason you don't put all your eggs on a single disk.
Anecdotally I have keys from 2013 that are still working fine when I test them, although they've been superseded by newer models.
To my knowledge no one has done the Consumer Reports / Mythbusters style of testing, buying many Yubikeys over time (to get different hardware batches) and subjecting them to various tests...
If you're using tmux on the VPS this can get a little tricky, because by default your tmux panes will inherit your initial SSH client's SSH_AUTH_SOCK; when you re-attach tmux in a different SSH session you'll no longer be able to authenticate with the local YubiKey. But I wrote a shell script that hooks into tmux to keep SSH_AUTH_SOCK working correctly with this kind of setup: https://github.com/mshroyer/socklink
ssh-add -s <pkcs11.so>
To mitigate that you can add -c to the ssh-add command. That will make the agent prompt for confirmation (on the host with the PIV key) every time something asks to use it. Doesn't quite get you the same security as just not caching the PIN to start with but it's OK.
i can't lie, the thought of removing authenticator/authy 2FA from my critical accounts is a little daunting.
They started selling the Nitrokey 3 long (like years) before the software had the main features they claimed they had, so I understand why they would want to be able to push updates. But again in terms of security, the first thing I want from a security key is that a compromised computer cannot reflash it...
Yeah. Precisely my problem with Nitrokey too.
If you can flash it, then so will a bad-guy one day. Not if, but when.
fido, gpg, piv all work perfectly
Something about stealing an iPhone jailbreak years ago?
the ios thing was in 2009. the company that produces nitrokey security hardware was founded in 2015
Whether you want or don't want that property is a different story.
I thought private keys were meant to be secret (not copied) and the public keys meant to be installed onto the target host.
so if you have a desktop, laptop etc, you would generate the secret once per endpoint and then copy the public keys onto targets as needed. AT scale this could be done by ldap / cloud mgmt.
For now, I'm hosting my own Vaultwarden and I pushed all my ssh keys to that. Anyone feel strongly against that? It stores their passphrase as well.
I add YubiKeys as backup everywhere I can. Like I have one YubiKey that always stays at home in case I lose my laptop and phone so I can get access back to some stuff sooner than later.
To answer your question, a CLI is a lot easier to write than a GUI. So the GUI would be the next step after the CLI, if there was enough demand. Now I'm guessing that people who care enough about security to try to handle it properly are anyway people who can use the CLI.
The only way I can get root (short of using a local privilege exploitation exploit to gain root) is by logging, using a Yubikey, over ssh, from a dedicated laptop that has nothing but a SSH client. So the Yubikey is plugged into the laptop.
That dedicated laptop has no wifi drivers and is on a dedicated LAN and that dedicated LAN only has two machines on it: my desktop and that laptop.
The laptop serves no purpose other than: "log in as root on the desktop".
Of course both that laptop and the desktop have firewalls.
This means, yes, I allow log in as root using SSH. But password logins are disabled and the only way to log in as root using SSH is to use the Yubikey. And the firewall only allow SSH attempts to port 22 from the MAC address, IP, and dedicated LAN that corresponds to the laptop.
In addition to that my main desktop has multiple user accounts: but none of those users are allowed to su or sudo.
Now, I know, I could use Qubes or whatever.
But what I explained above is how I do it: even myself I cannot get root on my own desktop without my laptop + Yubikey (I'd need to find and enter the 40 chars long root password).
When I really need to do something as root (and once becoming root becomes extremely hard, suddenly you realize how rare are the occasions when you really need to be root), I open my laptop and boot it (and because that laptop has no GUI and basically nothing besides a SSH client, it boots in three seconds or so).
Haters gonna hate but that's been working totally fine for me since years and years.
1. Virtual terminals? The answer I can think of is usb hid attacks.
2. Serial Consoles? My answer the is encrypted connections on the wire.
Any other reasons than mines?
I'd rather have qubes than hardware resident keys.
If you're using userspace software to decrypt the keys into memory then it's all just security theatre. Any sufficiently motivated actor, ot a comprised package in the software chain, can capture your private key in the clear. Especially if using ssh-agent that keeps it resident in memory 'for convenience'.
Proper HSM integration with SSH requires challenges signed by the HSM without the private key ever being emitted.
Just keep your SSH keys on disk, one per remote host, with a different encryption password for each target host. If you're worried about threats that can defeat that then they'll just come to your door and beat the passwords out of you.
Only because you clearly don't understand how it works. What's interesting is that you seem to understand half of it... or maybe you don't, and you just repeat stuff you don't understand either?
> If you're using userspace software to decrypt the keys into memory
That's exactly why you don't do it. A Yubikey or a secure enclave create the private key and emit the public key.