We've probably been to NYC ten times at this point. We stay out of Midtown except if we are going to a Broadway show. The subway can get you quite a ways. Head out to the Cloisters instead of the main Met museum. Be sure to walk around the rest of the park. It's a completely different version of New York. Not better, not worse, but completely new if you've only ever done Times Square and other popular spots. We took a subway and bus out to the Botanical Garden. Another gem that gets few out of town visitors but gives locals discounted access and they use it.

Heck, just walking around Harlem will give you an amazing day with 20% or less of the tourists.