It does, but modern heat pumps are so efficient compared to an oil-based system from the 1900's it'll save more energy than new insulation.

Had the house been uninsulated it'd be another story, but such houses are very rare in the Nordics.

Depends where you are, and what your start point is.

I've looked at improving old houses in Australia, and you would usually be better off putting the cash in the bank. (After heat pumps, and sometimes a battery).

Of course, but with a heat pump you get the combo of lower demand and much higher efficiency. In Germany for example due to high electricity prices you really need COP>3 or so to beat fossil fuel for heating economically.