At what point do we say matrix isn't going to work? It's been like this for years. It's so overengineered on an immutable chat history on a blockchain it has crossed the line into "way too expensive to run and maintain for streams of cat pictures and memes". Worse is better.
A decade ago Moxie Marlinspike argued[1] that to develop Signal more quickly, he and his team could not be distracted by federation. At the time I wasn't sure what to think, but it does seem that the effort level of using the integrated approach of client/server/protocol may just logistically be easier.
I'll happily move over to XMPP or IRCv4 if they can get it right, I'm not married to the way Matrix does things. But it's good enough, open source, and federated.