> The kernel's page fault handler is reactive — it doesn't know you're about to read layer 47's FFN weights, so it can't prefetch.
replyman 2 madvise
That works for readahead but it's not good for random access. readv, aio, dispatch_io are better there.
replyThis claim is a bit apples and oranges (no pun intended!). madvise is all about providing hints to the kernel to tune the page cache and readahead (including possibly disabling readahead altogether). it's not about performing reads into private memory buffers, which is actually where the options you mentioned fit in.
replyThat assumes you have significant work to do between fetches (so you can prefetch while using the current data). With LLM decode you don't.
replydeleted
reply