Dense architectures hit every byte of the model for every token. It's the ultimate memory bandwidth nightmare.
MoE models use subsections for each token and are more amenable to keeping parts in secondary storage, and indeed there are some local llm engines that do this.