10 months ago I've built a Raspberry Pi 5 mini NAS with 3 SSDs connected through RADXA Penta Hat. It runs OpenMediaVault, system is on the SDcard, other data are on the SSD ZFS pool. I have autosnapshots enabled, and I tune compression methods on specific datasets (lz4 for almost everything, zstd for file backups).

We use it for storing (backups, media files), playing (PaperMC), and watching (Jellyfin). I can only complain about the lack of hardware decoding in Raspberry Pi 5. Jellyfin loads CPU much if I enable transcoding so it's always disabled. If I knew this, I'd consider a cheaper and faster, but less popular, RADXA machine. Storage is fast enough for me, rsync and Samba speeds are usually limited by my network. PaperMC also runs without a hitch, thanks for asking!

As I didn't have high requirements for the machine, I also considered USB Bays at first but they wouldn't go well with ZFS.

USB/SATA adapters are incredibly flaky. Some don't support TRIM, some have major firmware bugs, many overheat during long data transfers and disconnect from the host. I'd strongly advise against using them for any important data.

I'd probably just buy a cheap prebuilt PC with a big case, or surplus enterprise tower. You can get cheap LSI PCI cards and do miniSAS to SATA if needed.

The biggest concern I'd have with USB is power delivery to the hard drives, but I haven't even done the napkin math so maybe it's fine. The SSDs seem like they might be a waste of money. USB hard drives have a poor reputation, but I don't have a ton of experience to say how much of that is deserved. On a practical level, I'd also be concerned about knocking the cable out.