This is more or less what those lovely 1.5 bit quantisations are already doing