The agent requirement is a symptom of a deeper potential issue: not using the system's ssh. Providing built-in as a fast-path option is fine, but ~/.ssh/config can do a lot of useful things, and providing an easy option to say "just use real ssh" lets people have their config just work.

Specifying identify files for specific hosts, using bastion hosts (ProxyJump), etc. For example, my laptop SSH config detects when I'm not on my university network or VPN and adds a ProxyJump through the department bastion host, but connects directly when I'm on the network. Configurations like that break when software makes too many assumptions about how ssh is configured.

hi - this is incorrect

I use a Yubikey for some of my keys, and I've never used an "agent", don't know how, and don't want to start, unless someone can give me a good reason to

ssh-keygen -t ed25519-sk

that's all you need to use a Yubikey with SSH

you don't need an agent for hsm.

also, don't know a good security researcher who uses agents (some even use ssh config to limit which keys are even tried per domain)

ssh config hacks are also a must if you have, say, two or more identities to github.com for example.