You don't need unprivileged user namespaces for this one if you're in a position to get the target kernel module loaded. But yeah, user namespaces are basically the single most significant privesc path in the kernel, maybe io-uring is second. Disabling both (or very carefully deciding what can use them) is one of the best ways to reduce your attack surface.

I don't have any guides but you can determine which kernel modules are already loaded in your system and then just compile those in and block module loading.

Otherwise, shove everything into a container, ideally gvisor, and you've reduced attack surface by a large chunk again via seccomp.