We are trading away disk space for faster builds. We could make them faster in some cases by using even more...
replyOn the other hand, it would be good to garbage collect those caches. We are wrapping up work on a new layout for intermediate build artifacts that will make it easier to GC them.
Sounds like what I was thinking, that or for third party deps to go into the same temp folder, and anything not accessed in over a week, or so just gets auto wiped by rustc / cargo build process?
replyCan you talk a little more about that? How would that work, and what would the developer experience be like when using it?
replyThe proposal's at https://hackmd.io/@rust-cargo-team/SJT-p_rL2; progress is tracked at https://github.com/rust-lang/cargo/issues/12633.
replyI always got a kick out of that, coming from a JavaScript background where people constantly harp on the size of node modules.
replyMy Tauri project, where the backend is much smaller code-wise than the frontend, has 9gb of rust artifacts (node_modules is 550mb for comparison)
Rust isn't great, and it shouldn't be a surprised since it's designed after npm. However one metric where nodes_modules is still worse for me is the sheer number of small files in it.
replyHaving nearly one million files in nodes_modules isn't that unusual. The problem is that on most common file systems the minimum allocation is usually at least 4KB. So even if the actual data is less than 500MB, you end up with 4GB disk space used/wasted.
I wish ext4 had a feature to mark a file as "atomic" where it would allocate all atomic files in a long run, without room for expansion, and I suppose with very inefficient compaction upon deletion, but without any padding bytes.
replyA file “pointer” for byte exact files, pointer gets ditched for files that get updated or the pointer gets adjusted to another common file.
reply