> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design.

No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returned, the process table entry was duplicated and adjusted so that there were now two processes, one in memory and one swapped out. The one in memory then got control, and could do an exec() call.

This allowed large programs to run on small PDP-11 machines. It was needed back in the era of really expensive memory. That's why.

QNX had an interesting approach. Program loading isn't in the OS at all. There's "fork", but program loading is in a library. It links to a .so file which reads the executable header, allocates memory, loads the program, gets it ready to run, and starts it. The program loader runs in user space and is unprivileged. This is probably the right way to do it.

It is somewhat interesting that the most widely used "big" OS that doesn't use fork, i.e. Windows, has dog slow process creation...

I agree that there should be non-fork primitives, I'm just not that sure that performance is the best argument.

This paper is great and I also really like one of its references [29] as it goes into some more subtle parts of scalable interfaces, including fork. It's a gem IMO: The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors https://people.csail.mit.edu/nickolai/papers/clements-sc.pdf

Discussion at the time:

https://news.ycombinator.com/item?id=19621799 - A fork() in the road (2019-04-10, 178 comments)

Fork is marvelous for the zygote pattern

Hard to come up with an optimization that is equally efficient and elegant

This was left implicit in the article, but what they mean by copying the process state here is the memory management structures. That's mainly the page tables and the VMAs.

That means you have to allocate new pages to hold a copy of all these structures, even if the actual memory pointed by the pages is shared. And walking all those structures to make a copy is still costly.

Even with copy-on-write, fork() still has to pay the setup cost for COW. If the parent process has a lot of busy threads (e.g. Java), you can end up doing a lot of unnecessary COW before exec() fires.

> It's weird to leave out a mention of copy-on-write

For the intended audience of such a paper this is base knowledge.

It says state. Copy on write still means it's O(number of page table entries) even if you don't copy the contents. It's a well known issue that forking a program with large virtual memory size is slow.

But you generally want to communicate with that process, so you do need to setup e.g. file descriptors and stuff, which needs information from the parent process to be passed.

What do you mean by "a completely new process"?

Isn't that covered by O_CLOEXEC?

>It feels crazy that we don't have a way to directly express the latter thing

Isn't that what posix_spawn is for?

I have the entirely opposite opinion. IMO a big mistake of the UNIXy model is that so much state is preserved across the creation of a process. For example, there are APIs to have a specific thing be fd number 4 so you can run a program and have it find that thing at fd 4. This is weird.

Windows, for all its many, many faults, did not use fork+exec and instead mostly has options for how one creates a process. It wasn’t done elegantly, but it was the right decision.

The new system calls described in the article have an extensible declarative command interface built into them to do things like close or duplicate file descriptors. Not opposed to it but it definitely stood out to me.

I agree. I think the current way is very nice to use (in c). I think the best way would be to have something similar to vfork() but not bound by posix rules. Then make the normal posix apis (close, setuid, etc.) act like the Rust “builder” pattern. Possibly giving them a prefix for explicitness. That way the “fill out a giant structure” people could have their wish and the people that just want a faster posix experience don’t have to learn an entirely new concept and api surface. It would be future extensible that way, too (just add more prefixed calls to the builder).

Whatever elegance fork(2) has (or doesn't) have, clone(2) has more.

Yeah. The right way to eliminate fork() is to make the usual APIs that modify process state take an explicit process handle, so the same APIs can be used to set up an empty process. They can also be composed in other ways, eg for IPC or debugging.

That's mostly papering over design mistake that most syscalls doesn't accept target pid. Otherwise you could just create suspended process, configure it with syscalls that explicitly take target pid, and start it.

When cores start needing more than 9 bits to be represented and RAM is in terabytes, many of the old assumptions need to change. Schedulers need to be implemented in userspace, RAM needs to be allocated in GB, not in 4k, io needs to require less round-trips between kernel and user space and NICs need to do a lot more work before the data reaches the CPU.

I’m using Emacs and various cli tools and while threads are nice to have, they can easily ramp up the complexity of a program beyond what is necessary. I much prefer the boilerplate of setting up a thread pool and tasks queue, rather than dealing with all the await/async syntactic sugar.

libgit2 exists. You could imagine communicating with some gitd over a pipe/socket but I don't know why that would be a good idea. Short of that you have to spawn processes.

It sounds like they're interested in the concept though, just not that specific implementation.

Fork/exec is great if you actually want the traditional copy of your process for some reason.

For launching something totally new, like the example in the article of some tool calling git, I think it does make a ton of sense to make something new.

Especially since I suspect that is by far the more common case. I suspect “I want a clone of me“ is relatively rarely used at this point.

These discussions were definitely had back in the 20th century too. The spawn model versus the fork+execve model has been an on-going debate since the time of MS/PC/DR-DOS.

The LWN article is incorrect in saying that it "must copy the entire process state (including memory) for the child process". There are some kernel structures and page tables that need to be initialized, plus you need a new stack, but it's not nearly as dramatic as implied. Most of the parent's memory is "incorporated by reference", so to speak.

In fact, if you profile it, in the fork() + execve() model, execve() is far more expensive, because not only does it replace the old process with a new one, but it also involves running the dynamic linker, which opens, parses, and mmaps library files.

It still makes sense to get rid of the fork() overhead if you're going to throw away the cloned process state soon thereafter, but if you wanted to make process execution radically faster, rethinking the exec architecture would probably offer more significant gains.

I'm a bit naive, but I don't think that's necessarily a requirement.

It might be commonly held convention, and thus, an assumption, in Linux (and, broadly, UNIX) but I don't think it's true inside VAX or even Windows, so I don't think it's a requirement.

Unless I've missed something (which is totally possible, this is not an area of OS design I've spent much time).

A lot of times you actively don't want the parent environment or any of the memory or file descriptors. And then you have to actively do work to fix all that stuff up after the fork.

the environment is not that big

Launching git repeatedly was probably not the best example. But it's hard to think of good examples where launching processes repeatedly is the most performant thing to do, probably because launching processes had been expensive and everyone has learned to do something else (libraries, zygotes, etc). Maybe a different question is: if launching processes were cheap, is there something we would implement as processes instead of libraries?

I can recall just one program that's intentionally not implemented as a library, but I think people have since built a library on top of it:

https://dechifro.org/dcraw/#:~:text=Why%20don%27t%20you%20im...

But when you use a process, you get tons of things for free, the subtask is invoked in parallel, you get isolation and you can control execution for free. Unless you are already writing a multithreaded program or already accept passing objects in memory, using a process is actually easier to write than using a library.

If I use a library, I also need to start using threads and need to invent some core synchronization mechanism. I essentially are reinventing a small scheduler, when I already get this from the OS for free. Also know any crash in the third-party code will crash the whole program, the third-party code has access to the whole address space. With invoking a process you also have a standardized API implemented by the OS.

There are lots of reasons to want to spawn fresh processes, which aren't solved by linking a library.

Shared libraries (and mmapped files in general) are deduplicated; it's nowhere near as bad as you think. The kernel loads a .so into memory once and then maps that memory into every process that mmaps it.

Editing to add: this deduplication is one of the greatest upsides to dynamic linking. Common libs like libgcc and libc only have to exist in memory once and can stay in CPU caches, whereas if they were statically linked into every binary, each binary would have a copy of that library that wouldn't be shared with anything else and you'd waste a lot of memory.

Typically libgcc_so.so is loaded by the linker, which uses an mmap call to map the binary into the address space.

> The kernel keeps track of which file is mapped where, and can detect when a request is made to map an already mapped file again, avoiding physical memory allocation if possible.

Relevant stack overflow answer: https://stackoverflow.com/questions/61950951/linux-shared-li...

In Linux, when a shared lib is loaded by multiple processes, its loaded once and not duplicated in ram. Only if a memory page is modified by the process will the memory be duplicated. (Hope I have explained that correctly)

Those mappings by default all go to the same shared memory.

Unices have been sharing executable memory between processes longer than there's been mmap for user space to do the same thing themselves. I remember seeing it in the 2BSD kernel for instance.

Eh? Aren't shared libraries actually shared in memory?

I have a rule for myself. If I think something is silly or stupid, I assume I don't understand it. I usually find I do not understand it, and it no longer seems silly when I do understand it.

In this case too, you think it is silly because you don't understand it. Your assumptions are wrong, making it seem silly.

It can also mean that neither the hardware side or the software side is static, but change over time. That means that their demands and what they allow also change over time. This leads to the insight that what was perhaps a good idea on 70s hardware/software is not necessarily a good, or even ok, idea 50 years later on modern hardware executing OSes and programs that have been kept up to date.