Yes, this is my experience for muscle at least. I used to work out 3-4 times a week, maybe a little more sometimes. Lately due to circumstances, I've been doing smaller workouts about 1-2 times a week. I've lost some finesse, but my muscle mass has remained roughly the same.

Also like some people hinted at this in sibling threads, I think it's different between purely abstract skills, and skills that involve muscle memory. For instance, I could probably stop using my bicycle for a very long time, and still not unlearn how to use it, or learn it again really quickly. Maybe it is because abstract skills are inherently more complex and require more cognitive effort and connections to knowledge overall, and are therefore more fragile.