Have you ever sat down and hand-done how to turn a number into a quantized integer, or done int4 multiplication by hand? It's not at all a normal integer.
And they're also not 24 bits of precision.
The only real reason GPUs have ever supported true integer math beyond stuff like tid/gid basics is for bitcoin acceleration, but that's nowhere near the speed of floats/quantized ints.