Who cares if they're doing exercise or not? The person who takes walks presumably knows it's a form of exercise. They're not talking about the other forms, they may not be able to do Crossfit or go skiing, and they might not feel confident expressing opinions about the entirety of all exercise, but they definitely know that walking works for them.
I do somewhat see the value in promoting specific, accessible meditative practices without necessarily using the word meditation for it, simply because it can be needlessly intimidating and put some people off because they come carrying a number of assumptions.
Maybe that same principle does also apply to exercise - some people will do it by accident and have a good time, but still balk at idea of doing capital E Exercise as a distinct activity in itself. Sometimes it really is just a mindset thing.