I'm not denying that X11 has known security issues. However, I don't tend to run untrusted random gui applications on my system so that factors into the risks I'm willing to accept.