Sounds like there is some foundational knowledge of Elixir that you miss and everything seems more confusing than it should be. To me writing a 'server' in Elixir is orders of magnitude easier than doing it in Python, Rust or C++.
As someone else suggested, bring your concerns to the Elixir Forum and surely someone will clarify them for you
OMG, why? Why would you ever have so many processes? All of them at the same time? Are you going to animate a 3D scene and run a process for each vertex, or something?
No, I mean, if you're WhatsApp - across all nodes - then somehow maybe yes? At scale. But in normal code, slicing workloads too thinly is counterproductive, and having even tens of thousands of processes is a sign that you're slicing it way too thin. Message passing between processes is cheap, but not free. Schedulers do a good job, but rarely have more than 16 cores to work with. And so on.
You can have that many processes if you want, to be sure. But if you're struggling with it, why would you want it?
Reading your comments in this thread, I have a feeling you just didn't spend enough time reflecting on how you want to use Elixir. In effect, you also failed to consider how exactly you should learn it. For example: Elixir is a perfectly capable procedural language. Start by writing CLI tools, without spawning any processes at all. Then try to parallelize their processing. If the tool accepts a list of files as arguments, use a `Task` to compute return values for each file. Tasks are processes, but with a particular contract that simplifies their usage. Later, you can experiment with error handling and supervision by putting the tasks under a supervisor. And so on. You go from the familiar to the less familiar, with a useful, working tool every step of the way.
I mean, we had one process per client connection (which is 100% the way to go) and depending on the era, hundreds of thousands or millions of connections per chat node. I don't think we ever really summed the number of processes over a cluster.
Other than client processes, there weren't that many processes per node; like you say, it doesn't make sense to spread too thin.
There's a lot of client connections and so a lot of client processes, but it ends up being pretty simple to work with them. They all do the same thing... wait for a message, process the message, wait some more. Some of the messages are tricky to process (like the user just logged in again over here, so please transfer the state)
That's a bit of a misrepresentation. Error handling on the BEAM has a few more layers than in other environments; specifically, the supervision tree can be used to "let things fail". That's not the layer where you should log or handle failures - that's a safety net that ensures your whole system won't go down if your error handling in a single process doesn't work.
For error handling, there are roughly these layers:
- functions can return {:ok, value} or {:error, error}
- functions can raise errors (similar to exceptions) that can be caught
- processes can be monitored from the outside, you get notified when they die
- processes can be linked and exits can be trapped, also notifying you on failure
- supervisors can handle process deaths in a configurable manner
- higher-level behaviours often expose their own error handling callbacks
That's a choice, but it's not idiomatic.
You're expected to write things like...
ok = thing_that_might_not_work().
Let it crash ends up not quite working, so you end up catching a lot of errors, but you should be logging them, not swallowing them...