upvote
The NUMA use cases exist, but given the rarity, I'd rather the default behavior not prioritize them. Currently you have to explicitly tell the system not to let a task use multiple nodes' resources, not just CPU/memory but also PCIe. Maaybe it's smart enough to avoid some cross-node usage, but it's a soft heuristic rather than a clear rule, which actually makes it more of a footgun than being dumb about it.

I would rather it be the other way around, never allow a single process to do anything cross-NUMA unless it asks for that, maybe even be stricter and require a process to opt into using anything but node0. These machines are big enough that you're not going to saturate node0 with random tasks, and you're only going to saturate the whole machine with a more deliberate workload.

reply
AMD's NPS4 mode isn't exactly user-friendly, I agree. But you can put it into NPS1 mode and relax. Graviton 5, as a counterpoint, doesn't give you the option. Physically there is a 2D mesh between the cores and the memory controller but the observable behavior is that every access gets the average mesh fabric latency. The efficiency you leave on the table isn't very large, whereas in multisocket NUMA you can't ignore the cost.

I think you can over-analyze this stuff and lose your sanity. On these multicore systems there are also hot cores in the center of the mesh and cold ones at the edges and theoretically you could be doing temperature-aware scheduling, gaining a bit more efficiency in doing so. But it's just easier to adopt the black box model of spherical frictionless CPUs.

reply