You're about to respond: "But many of these use Android, and general purpose computers are too distracting for me." In that case, you'll need to forego live internet audio streams and buy a closed option with a radio receiver.
Get a second-hand Apple iPod Touch, remove all apps you don't need.
For just mp3 and podcasts: get an iPod Classic (or Video) and install Rockbox.
Rockbox is amazing.
That internet radio is a whole magnitude of complexity, especially with the need for wifi (cellular?) if it needs to be portable. But there are options like specially modified android devices.
I have the Shangling M0 with a 512GB card. I don’t even bother with converting my flac files. A nice other device is my kobo. It holds my entire fiction library with space to spare.