Basically. Unity video player is fine if you control the content. The other very reasonable option is to funnel shared content through a synced web browser interface. I've developed a few collaborative XR applications, and have used a blend of things to support sharing.

It’s not hard to use libavcodec to transcode, though.