Of course, it would be far better to just not have the memory leak...but in the mean time, macOS memory compression is making the leak nearly a none issue.
Sure, they will get restarted, but not great.
going through extra effort to accommodate/enable buggy, poorly behaving programs, and meanwhile—
punishing those (and the programmers of those) that are well-behaved and functioning exactly as intended but that now have to contend with evading some heuristic of badness.
Thus solving memory leaks forever...
Fun days.
Now, we find out when the SSD wears out, in the worst case.
Since the models are effectively highly compressed data already my assumption is that RAM compression won't help improve that much (I'm assuming unified memory here as per the Framework Desktop). A bit like trying to compress a JPEG in a zip file.
Could anyone comment on that?
Most quantized GGUFs have a very high Shannon entropy, so are almost perfectly compressed already. A Q4_K_M quantization is often around +7.90 bits/byte and so is virtually incompressible.
Unquantized models (F16) are often around +7bits/byte for comparison.
https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...
I'm not too familiar with LLM memory access patterns, but since the model files are stored on disk, you wouldn't ever keep them in memory anyway (except caching) and just reread them on demand.
Dense architectures hit every byte of the model for every token. It's the ultimate memory bandwidth nightmare.
MoE models use subsections for each token and are more amenable to keeping parts in secondary storage, and indeed there are some local llm engines that do this.
One way to think about it is that if the weights were compressible, it’s a clear signal that better training could make the model smarter as there’s redundant information in the structure that could be used to store other meaning.
The other challenge with compressing weights of course is that you have to decompress them before you do math with them. Dequantizing weights is significantly cheaper and supports random access too (not that that matters too much but it helps in terms of leveraging SIMD techniques to do it in an embarrassingly parallel way)
Consider the induction heads, their W_O are generally expected to be the identity matrix and it is useful to have many of them at many layers (e.g. if you only had one induction head in the whole network you'd only be able to perform one induction circuit per token emitted)
That being said the attention heads are a small portion of the total network size
Interesting comparison, I wonder if there are loss-less compression schemes that are "weight-aware", about the statistical properties of model weights.
Back to that line - because (Huffman coded) JPEG images can be lossless recoded using arithmetic coding, which makes them smaller than if they were compressed with the best general-purpose compression algorithms/tools (e.g. `jpegtran -arithmetic < input.jpg > output.jpg` beats `xz -9 < input.jpg > output.jpg`). AC takes advantage of the properties of the image coding better than general-purpose compression tools on the output of the Huffman coding.
But all seem to have massive issue boards. What I did see was oMlx release clustering so you can string 3 macs and have 3 times the memory or use one for prefilling accelerating.
Which I thought was neat and what I'll likely poke at instead of getting more ran.
So my mind on that.
Yes, macOS has had it enabled by default since OS X Mavericks 10.9, released in October 2013. I’m surprised some desktop Linux distributions still require manual configuration to enable it.
For browser tabs, though, I think a better solution is automatic tab closing, using something like the Tab Wrangler extension for Chrome. Collecting tabs comes naturally; we’re much worse at closing them.
In Firefox there's about:unloads where you can suspend tabs in I believe LRU order. I'd really like to see this as a default option (like: only keep N tabs active). Maybe there's an extension for that.
Do Shift+Esc and watch how large the GPU memory usage is. Often 2/3 to 50% of your actual tab usage.
Ironically, its not the closing of tabs that bothers me, but the fact that tab order changes, what slows my productivity/muscle memory down. And its the same reason for a lot of people why they keep tabs open.
I never understood why we do not have a basic snapshot / sleep that offloads a tabs memory on disk (and dumps the GPU memory), aka a ghost tab. And fast loads when you view the tab. Sure, there may be a small delay but it can save so much memory, while people can keep their tabs open/organized.
Some extensions try to mimic that but its often a crude solution because they do not have deeper access to the webview2 engine.
Its a user choice and should be a user-facing knob because there are times it is inappropriate for use, such as in safety-critical circumstances, certain realtime realities, and so on. It should probably be on by default for desktop Linux, but off by default for every other application a distribution may be targeting ...
[1] https://en.wikipedia.org/wiki/SoftRAM
[2] https://devblogs.microsoft.com/oldnewthing/20211111-00/?p=10...
If my understanding is correct, both ZRAM and ZSWAP write uncompressed pages to disk. And both don't pack the pages together meaning even if we'd write compressed pages (let's say they compress 5:1), we'd still write the original size to disk. This is crazy.
ZRAM added compressed_writeback in 7.0, it needs to be explicitly enabled.
I recall reading an article around the time the Windows team added it (back when I still used the OS). Their compression algorithm was typically faster than the latency to the prevailing disk type at the time (SATA SSD, most likely) - at the time it had zero downsides. I'm not sure if that still holds true.
On Linux it generally doesn't: off-the-shelf compression is used, which do not outpace storage access latency.
https://chrisdown.name/2026/03/24/zswap-vs-zram-when-to-use-...
All you're doing by running without swap is pessimizing yourself.
if you don't want to swap to disk you should not use zswap.
zram is ideal in that scenario, though it may complicate OOM killer considerations (you may want a userspace one).
I am hoping to switch my dad from ubuntu to fedora in a month or so because he has 16 GB ram and this seems like something most people dont know.
I really like Fedora in that it seems very up to date package wise but I found it requires a reboot every time I update using the UI updater.
It's better using the dnf command but it still felt a bit lumpy compared to apt.
I find that the ratio of browserish memory use to that of other programs is enough to render the latter insignificant most of the time.
Isn't that wasteful? The CPU's caches may help a bit, but I would guess first a huge block is going through the decompression, so only the last bits are still in the cache, when the actual usage starts.
If it avoids hitting the disk I get it, but to some of us, that just means it's time to buy more RAM
What's interesting to note is that I've occasionally wondered why isn't the swap partition compressed.
If you have it on an NVMe drive it probably is as is all data! The controller usually transparently does it as it writes to/from NAND.
So the kernel sees uncompressed fixed block pages and the speed boosts happen with data arriving faster when read or written to flash.
I started using zswap recently but didn't have the time yet to measure the benefit. Does it benefit machines with low amounts of RAM and wimpy CPU (like a RPi3)?
> The controller usually transparently does it as it writes to/from NAND.
In the consumer SSD space the last controllers I know did on-the-fly compression were SandForce [1], probably 15 years ago, when minimizing writes and saving the little capacity available was imperative. I don't think any other desktop SSDs do this now. A bit more common in enterprise models like ScaleFlux CSD.
Tried porting it to swift, was hoping it would make mobile local models more useful for a project I was working on but it was hurting the accuracy too much. Can be very task dependent on how much you can compress. https://github.com/nburns/llmlingua-swift
My specific use case was using a LLM on segments of labeled data from a different faster model as a final pass.
It sounds like those are downsides
I did not really notice a large performance impact.
Unless you are in an environment where compute performance is critical, I think this is something interesting to try.
It should be a user switch and probably will become one in most Linux desktop distributations, I would wager. However, I would not enable it - I run Ubuntu Studio for every last little bit of latency it gives me in a realtime audio-capture application, and on those systems were it to become default-on by way of distro update, I'd be kinda miffed.
It’s like saying that in order to breathe you have to use your nose or mouth. Which is bad if you want to retain the oxygen that is inside your lungs.
There are always scenario’s where memory compression is an outright bad idea. Just like trying to breathe under water.
Which one is it?
It's a remarkable tech that deserves more attention. Swap is back in style!
Instead of buying more RAM, which meant that I should replace my device, I decided to implement zram. This compresses memory. Now I can do more with the same hardware.
In many cloud environments compression is not enabled. I am curious to see if the same gains can be made in such environment.
There might be gains by having the GPU recompress with a higher compression level in the background though.
So shouldn’t disable pagefile, unless it’s a disk space issue. Windows probably won’t use it much anyways if you have enough RAM.
SSDs make swapping less of an issue, but it's still something I'd rather not allow to ever happen, as it's difficult to get swapped-out pages back into memory without exercising every single application on the system.
I thought memory compression was a replacement for swapping entirely, because you don't need to evict from memory, you can just compress it.
DiskDoubler, Stacker, SuperStor, DoubleDisk...
That’s true of everything. Default setups make those choices.
For a server, it really depends on what kind of service you're providing whether it would be helpful. We might want to let server admins, SREs, production engineers, or platform engineers to make an active decision about it.
There are plenty of scenarios in which servers are run with only nominal swap sizes or none at all and systems don't run out of RAM.
Let me size my hypervisor cluster to demand and let the system decide where to place the guests. If the hypervisors are crashing due to full RAM, that's down to planning or budget issues and not a reason to put additional unpredictable latency underneath the guests.
Let me size my SDI to UDP TS video streaming servers to the number and size of streams they'll service. There's not much in their memory pages that's going to ever be idle enough to compress, so why boot the kernel configured to do that? It's just risking extra packet IAT for buying not much benefit.
But in general, yes, zswap would be a good thing compared to swapping to a drive, unless you are mostly swapping things that are really truly idle a long time. Default swappiness on a lot of distros is around 60 by the way, and on servers it's often set closer to 10 or 15.
Technically yes, practically on Windows memory compression is not used.
I have Windows 11, 128 GB RAM, and my memory stats show: 42 GB in use, 380 MB compressed.
I'm a Linux user myself, but I think it would start compressing it when there would be less than 25% free
That's for RDB persistence. https://valkey.io/topics/persistence/
I wonder if there's any similar wins with ram? That some workloads would see net wins from compression. Some cost of cpu, but increasing net throughput significantly.
As usual, working class is being told to tighten the belts whilst the rich get insanely richer.
Fuck that.
Still perfectly usable and I won't upgrade until prices come down. Even used prices are terrible now.
The main thing I regret is not maxing out the memory on my server. At first 32Gi seemed more than I'd ever need, but now it's feeling quite anaemic.
Typo DDR5?
https://www.intel.com/content/www/us/en/products/sku/236783/...
> If in doubt, prefer to use zswap. Only use zram if you have a highly specific reason to.
> In terms of architecture:
> zswap sits in front of your disk swap, compresses pages in RAM, and automatically tiers cold data to disk. It integrates directly with the kernel's memory management and distributes pressure gracefully.
> zram is a compressed RAM block device with a hard capacity limit. When you put swap on it and it fills up, there is no automatic eviction, and the kernel has very little leverage to do anything about the situation. The system either OOMs or falls back to lower-priority swap, causing LRU inversion (see below). It only really makes sense for extremely memory-constrained embedded systems, diskless setups, or cases with specific security requirements around keeping private data off persistent storage. Swap on zram is also increasingly unsupported upstream.
( https://news.ycombinator.com/item?id=47500746 )
It’s long and more satisfying than this In These AI Times wrapper.
I would rather have the system tell me there's not enough memory than try to pretend there's an infinite amount of it and then slow to a crawl without apparent reason. Unix memory allocation is daft. Desktop swap to block never worked, never will.
https://linuxblog.io/zswap-better-than-zram/
https://chrisdown.name/2026/03/24/zswap-vs-zram-when-to-use-...