Some 5-digit zip codes represent the PO Boxes at a post office building.

Some zip codes represent a single building. Example 20500 is the White House. The Pentagon is so big it has several zip codes.

https://convene.com/catalyst/office/buildings-new-york-city-...

It used to be that the first 3 digits of a zip code sent that mail to the "main" post office building (the one that has loading docks for like 10-30 tractor trailers on one side of the building). Even if you were mailing a letter to the same zip, it went to that building to be sorted and returned to the local office before delivery. Nowadays, everything goes to central facilities where stuff is scanned and sorted. In Kentucky, almost all mail goes through Louisville.

A lot of people also don't know that your written address is also disconnected from municipal government boundaries.

The "city" on my address growing up was a CDP[1], and it is now a city that I live 2 miles from rather than the one that I live less than a mile from.

1: https://en.wikipedia.org/wiki/Census-designated_place