In some countries and cultures the good of the many comes before the good of the individual, no matter the sacrifices required to get there. The US (and to a slightly lesser degree, Canada) generally does not subscribe to such a cultural mantra.
Edit to add: for the cases where the societal good really does outweigh the personal good, we have a tool for that: eminent domain, which is how we build train lines and so forth.
This line of logic doesn't follow. There's plenty of old people who need to move out for non-financial reasons (e.g. can't live independently anymore), and it doesn't really make sense to me that having to move out because you can't afford your mortgage is okay while not being able to afford your taxes isn't. In any case, we can have our cake and eat it too by building more housing. Homes become affordable for young people if supply increases and the tax base broadens enough that taxes go down for established homeowners.
Thus, you can budget around the mortgage payment in a predictable way, and if you have to move out because you can no longer afford it, that's a very different thing than if your property taxes magically change because a theoretical buyer may theoretically be willing to pay some astronomically higher cost for your house than you paid for it.
So, respectfully, I disagree with the framing that we have to take both or neither. They are separate budgeting concerns, and separate policy concerns.
Like another poster said, things like pensions and medicare might overall drain the entire economy, so the end result is the same, but that seems more diffuse than local tax bases, and wouldn't properly appear in local government budgets.
They're also economically and socially viable participants in the community.