Maybe PIM will push this forward, but I still think we're doing something fundamentally wrong by not just embracing NUMA and trying to do something Sun tried decades ago, which is have number of cores share 4GB of semiprivate working
memory.
replyWe've kind of half-assed it with DDR memory banks, but it mostly introduces mysterious slowdowns that are difficult to reason about and I think we would be better served I think by making a formal thing. Instead of introducing an L4 cache we could do this instead, and reduce the size of the L1-L3 caches, which shortens lookup time and thus latency.
For legacy apps, you could provide facilities for the OS to 'page' blocks in from main memory, but the speed would come from managing the workload imperatively, starting loads in the background before the data is actually needed, and dumps after it is last touched.
why would it? the parent OS can already handle physically contiguous allocations so these should be no different (with the exception that a separate interface can be used to do compute over these buffers/pages).
replyIf it requires physically contiguous RAM to work, then it's not really participating in the full virtual memory system, really. It would be using an exception to it, that can be accommodated to some extent by the OS, but not sitting in demand-paged storage like the rest of the system.
replyYou could still do map-reduce operations, but for it to really fly, what you'd want is a side channel between the memory chips that allows the reduce to happen out of band from the front-side bus and the reduction to be sent to the CPU. And any workflow where you can stream the reduction to the CPU that would be even better for latency.
replyThat really doesn't make sense because your system is already using memory that needs to be physically contiguous but mapped virtually. Also, would you say pinned memory is not part of the virtual memory system??
replyYou're only restricted by the fragmentation of the system memory which is an issue yes, but it's dealt with in other ways.
And the security folks will have tons of fun with this enormously leaky abstraction too.
replyHey, just don't store your pagetables in PIM capable sections :)
replyLinux has been dealing with this kind of thing for over a decade now. Specialized SoCs love their memory carveouts.
reply