Your understanding of the world "garden" probably came from 5+ encounters with the word in different contexts, and you cannot separate them - instead your understanding is some kind of average of the knowledge from each encounter.
Interestingly, adjacent and overlapping-meaning words like park, meadow, yard, grassland etc often have disputed boundaries - ie. One person might call something a garden, whilst someone else calls it a meadow. Neither can explain quite why they have chosen their specific word for the object in question, but it might stem from the source of where the meaning of each word was initially learned.
The Saussure idea of the differential nature of language seems relevant here too: words aren't just lists of things you look up, they are connected to each other and distinguish themselves by being slightly different from the other elements of the clusters they belong to.