Most people without absolute pitch have some level of "pitch memory", but it's not comprehensive or reliable. For instance, if you ask people to sing a pop song, they're significantly more likely than chance to sing it in the original key.

I know the Super Mario Bros. theme starts on an E, so I can identify an unknown pitch by recalling that theme and comparing using relative pitch. But that's quite a slow and unintuitive process, and it's easy to make a mistake. People with absolute pitch just hear the pitch without having to "recall" a reference note to compare to like that.