It's not necessarily that people forgot that reason for property taxes existing. Many voters fundamentally disagree with that as a valid reason. This is a matter of ideology and values as much as economics, and not everyone shares your priorities. You can label them as greedy hoarders or whatever but that won't change how they vote.
I have a 4500 sqft home with 6 bedrooms because I have kids a home office a recording studio and workshop. My Nextdoor neighbors are retired and have adult kids that moved out. Their house is the same size. They could downsize and make that inventory available for other next generation families. Instead they hoard it with a mortgage that is long since paid off and thousands of sqft that serve no purpose. They don’t even heat or cool the whole thing. Now extrapolate that across the whole economy. The obvious supply and demand distortion explains part of the reason homes are as expensive as they are.
You are correct on the technicality, but I think OP's point is that, in combination with NIMBYism heavily restricting supply, this is a form of consumption that imposes an externality on others.