upvote
Perhaps I'm not understanding the Matrix model, is there no central broker that is holding user contact relations and fanning out and directing statuses / messages?

Because, otherwise, even if you went crazy and did the most naive implementation:

    Storing for every User
    User -> List <User Contacts>
And the reverse: User -> List<Users who have you as contact>

And even then if you go full quadratic and have literally everyone have everyone else as a contact, and store and update the online status for each pair, and do that for each direction, you still shouldn't have scaling issues at 50k users, as that just isn't that much data?

By 500k Users you might be starting to feel it with a naive approach, but 50k? It shouldn't be a bottleneck unless status updates are causing cascade effects into other systems.

And that's just an analysis of the most naive design I could think of!

reply
I don’t believe there’s a central coordinator (it wouldn’t be distributed then).

I think you’re missing that rooms are attached to servers, and you can join rooms that aren’t on your server.

If I go join a room with 50k people on another server, those 50k people need to pull an online status from my server and my server needs to pull all of theirs.

Multiply that by all the rooms 50k people have joined and you get a _lot_ of messages bouncing around.

It’s sort of hard to optimize without either a central and implicitly trusted cache, or giving up on trust entirely. Central cache is sort of obvious. You could go peer to peer but then you have to deal with “what if somebody modified their server to issue forged updates”. You could maybe sign them but I’m not sure how anyone would validate that without getting a pubkey from your server and now we’re back to scaling problems and have a new “storing a million pubkeys” storage issue.

reply
Based on live metrics from my current server, there are three order of magnitudes more presence updates processed than any other form of event (typing, message, encryption, etc). This is after the majority of people have turned it off. It's a huge amount of work relative to everything else in the protocol.
reply
based on experience with another tool i saw, each lookup itself must be authorized by checking membership in at least two other lists on three other servers :P
reply