For Gentoo, of course, "just recompile the kernel as desired" is more reasonable, though they have binary packages including for the kernel and I don't see why the same idea shouldn't work there.
But I don't want to know what drivers I need and will need next. Tomorrow I could buy a different wifi module and then what? Spend 3 hours googling which rtl378326973268632aahaxhabt.ko to install? Thanks but no thanks.
We can have security and convenience.
It would work for various other drivers though.
You can do blacklists easy enough if you want to, just add few lines of text into /etc.
I'd also like option for whitelisting, like whitelisting every single NIC driver is harmless enough coz they just won't be loaded, but anything that can be loaded by non-root userspace action should have option to be only loaded if it is on whitelist.
Tho all that is easily doable by just changing userspace AFAIK