There's probably a few lower cost things that I am overlooking, to the tune of netting out a few hundred dollars of savings after however many years they took to pay back.
In the UK, houses have energy ratings, which are largely not that useful, but they do allow estimated annual running charge.
The house that I live in we moved in and were spending ~1.7k on gas a year.
We needed to re-render the place, because it has a few missing pieces. we spent the extra £4 to put in 90mm of external wall insulation. We also had to replace the glazing. It was cheaper to get triple glazing (for some reason), however the results of that was that it was 6degrees warmer in winter, and 10 degrees (celcius) cooler in summer. Even with gas prices doubling, we spend about £70 on hotwater and heating.