This assumes that LTO is strictly better than no-LTO, ie only gets faster, has the same optimization hotspots, and doesn't break anything.

I would recommend only doing things that fit within the 'build > text > fix' loop.