Look at a flyer on the wall, or your beverage if you’re in a bar, and they’ll follow up if they want to talk and appreciate the reduced pressure either way.
And yeah, never open a conversation with something like “can I ask you a question?” which is usually a trick of a salesperson or beggar to make you acknowledge them and start saying yes.
New Yorkers have a reputation for being stone cold with strangers, but the truth is that anytime somebody approaches you out of the blue, there's an assumption that they're about to ask for money or try to get in your pants. Once you demonstrate you're not looking for either (or, if the second I suppose, that you're at least smooth enough for it not to be immediately evident), people are generally really kind. With some exceptions, I've usually found that the coldest looking person will stop to give a lost tourist directions if it's clear they're in need.