foobar2000 has this!
After trying dozens of music clients (including PlexAmp) I eventually settled on simply using foobar2000 pointed to an smb share - nothing else seems capable of properly handling playback by folder structure. (e.g. I've got all my Led Zeppelin albums in the "Led Zeppelin" folder, and I want to just select it and play their discography in album release order.)
I find the mainstream subscription players to be excruciating to use to play what I want at the time.
I don't mind lacking features in the server or the client honestly, if I can find and play an album or playlist I'm good. But the connectivity piece can be hard if you're not familiar with self host stuff. I tried to do Jellyfin and Tailscale and some client and it... kind of worked? But it was definitely not as seamless.
I do hope every year that it's my last year paying for Plex. Maybe this year!
Because yes, the remote stuff -- and PlexAmp just working the same wherever I am -- is the huge part of it. Music sits on my NAS and things... just work.
I also really like how easy the caching and offline mode is, because I frequently find myself driving in areas without mobile data service. I keep a couple favorite albums downloaded to my phone, then knowing the app will cache the next ~2 albums of queued tracks generally makes it Just Work.