That’s why you see “base” vs “instruct” models for example — base is just that, the basic language model that models language, but doesn’t follow instructions yet.
Especially the open weights models have lots of variants, eg tuned for math, tuned for code, tuned for deep thinking, etc.
But it’s definitely a post train thing, usually done by generating synthetic data using other models.