Color depths, not modes per se, and it's more that XFree86/X.Org by virtue of being lowest common denominator didn't have optimizations based on it, and over time GTK and others forgot they exist.

Some systems like Xsgi went further with having several visuals used for predefined purposes in their modified Motif versions

I used X11 on some 8-bit framebuffers back in the day, and I remember the palette changing depending on which window was in the foreground. The foreground window would have proper colour, and the background ones would generally have crazy clashing colours.