And things like GSQ-RCO optimization from ISTA labs, EXL that you linked. Preserving bits for certain (large) domains like agentic coding. Pruning others. It really works and creates functional models.

I recently did some toying around with (very) small language models (<200k). I used QAT from the beginning and was surprised that I got byte identical results from the int8 model vs the float32 one.

If that scales with parameters I don't see why QAT shouldn't be the norm.

(As I was only comparing the QAT f32 model vs the quantization its possible that a not QAT f32 would have performed even better)