I will personally almost always prefer the pretty functional versions of things, and that's almost always what I start with. I like immutable data structures, and they are usually more than fast enough. Occasionally, though, you hit a bottleneck of some kind (usually in some form of loop), and you have to avoid all the beautiful functional stuff and go back to sad imperative stuff. When I do that, I usually try and keep it scoped to one function. Even within one function, I do find the persistent structures easier to reason about, but as you stated it's a small enough surface area to not be too irritating.

There are exceptions to this, of course. Sometimes for caching/memoizing I will make a global ConcurrentDictionary, and I'll use the interlocked thing to do global counters sometimes.