The rest of your post sounded rational. This part doesn’t.
Failure to build more units (NIMBY) and properties being held as investments (non-US nationals are significant in some markets) are the two causes I would point to. Of course every additional family that needs housing drives up the cost a little, but I really don’t think that can be responsible for the massive increases in housing valuation.
The other big issue is density is expensive; it costs much more to build an apartment at the 10th floor than a detached house. Combine that with density being inherently a bit shit, you get less quality and less housing for the money spent.