upvote

points

by Panzerschrek19 hours ago |
comments
upvoteby hrmtst938372 hours ago|
next
[-]
Those huge plugin files result from Rust duplicating libstd into each plugin at link time when crates are built as cdylib or staticlib, and you can prove it with cargo-bloat or by inspecting DT_NEEDED and symbol sizes with readelf -d and ldd.

If you control the build, force shared std linking with RUSTFLAGS='-C prefer-dynamic' and crate-type = ['dylib'] so host and plugins share libstd and accept the need for matching toolchains, or switch to a different plugin model like compiling to WASM and running modules with wasmtime or exposing a small C-ABI shim into a single shared runtime, and use strip plus LTO for quick size wins.

reply
upvoteby whytevuhuni17 hours ago|
prev|
next
[-]
Interesting, although I checked and on NixOS the binary is just 29MB. It was statically linked, with just libc left as dynamic.

I think 29MB is still huge for a terminal text editor, but nevertheless not "hundreds".

reply
upvoteby f311a16 hours ago|
parent|
[-]
Language grammars are ~200-250MB though. They are in a separate folder, and often they are all bundled to support all the languages. Some of them are HUGE.

  .rwxr-xr-x  4.6M aa    6 Mar 21:52  ocaml-interface.so
  .rwxr-xr-x  4.6M aa    6 Mar 21:52  rpmspec.so
  .rwxr-xr-x  4.9M aa    6 Mar 21:52  tlaplus.so
  .rwxr-xr-x  5.1M aa    6 Mar 21:52  ocaml.so
  .rwxr-xr-x  5.1M aa    6 Mar 21:52  c-sharp.so
  .rwxr-xr-x  5.3M aa    6 Mar 21:52  kotlin.so
  .rwxr-xr-x  5.4M aa    6 Mar 21:52  ponylang.so
  .rwxr-xr-x  5.5M aa    6 Mar 21:52  slang.so
  .rwxr-xr-x  6.1M aa    6 Mar 21:52  crystal.so
  .rwxr-xr-x  6.8M aa    6 Mar 21:52  fortran.so
  .rwxr-xr-x  9.2M aa    6 Mar 21:52  nim.so
  .rwxr-xr-x  9.5M aa    6 Mar 21:52  julia.so
  .rwxr-xr-x  9.9M aa    6 Mar 21:52  sql.so
  .rwxr-xr-x   16M aa    6 Mar 21:52  lean.so
  .rwxr-xr-x   18M aa    6 Mar 21:52  verilog.so
  .rwxr-xr-x   22M aa    6 Mar 21:52  systemverilog.so
reply
upvoteby Panzerschrek16 hours ago|
parent|
[-]
That's exactly what I found. Why these files should exist at all? Some other IDEs just have a bunch of highlighting rules based on regular expressions and have a folder of tiny XML grammar files instead of a folder of bloaty shared libraries.
reply
upvoteby homebrewer16 hours ago|
parent|
[-]
Because it's far more reliable to use proper parsers instead of a bunch of regular expressions. Most languages cannot be properly parsed with regexes.

Those files are compiled tree-sitter grammars, read up on why it exists and where it is used instead of me poorly regurgitating official documentation:

https://tree-sitter.github.io/tree-sitter

reply
upvoteby f311a16 hours ago|
parent|
next
[-]
Funny enough, they are less than 10MB when compressed. I guess they could use something like upx to compress these binaries.

The whole Linux release is 15mb, but it uncompresses to 16MB binary and 200MB grammars on disk.

Why do we need to have 40MB of Verilog grammars on disk when 99% of people don't use them?

reply
upvoteby homebrewer16 hours ago|
parent|
next
[-]
That would waste CPU time and introduce additional delays when opening files.

They could probably lazily install the grammars like neovim does, but as someone who doesn't have much faith in the reliability of internet infrastructure, I'll personally take it...

Just ran `:TSInstall all` in neovim out of curiosity, and the results were predictable:

  ~/.local/share/nvim/lazy/nvim-treesitter/parser
  files 309
  size 232M

  /usr/lib/helix/runtime/grammars
  files 246
  size 185M
If disk space is important for your use case, I guess filesystem compression would save far more than just compressing binaries with upx. btrfs+zstd handle those .so well:

  $ compsize ~/.local/share/nvim/lazy/nvim-treesitter/parser
  Type       Perc     Disk Usage   Uncompressed Referenced
  TOTAL       11%       26M         231M         231M

  $ compsize /usr/lib/helix/runtime/grammars
  Type       Perc     Disk Usage   Uncompressed Referenced
  TOTAL       12%       23M         184M         184M
reply
upvoteby f311a15 hours ago|
parent|
[-]
I mean, they could decompress it once when using a language for the first time. It will still be fully offline, but with a bit uncompressing.
reply
upvoteby e12e12 hours ago|
parent|
prev|
[-]
If this is a concern, why not compress at the filesystem level?
reply
upvoteby Panzerschrek12 hours ago|
parent|
prev|
[-]
For real parsing a proper compiler codebase (via a language server implementation) should be used. Writing something manually can't work properly, especially with languages like C++ and Rust with complex includes/imports and macros. Newer LSP editions support syntax-based highlighting/colorizing, but if some LSP implementation doesn't support it, using regexp-based fallback is mostly fine.
reply
upvoteby small_scombrus17 hours ago|
prev|
[-]
My local build of helix is 20MB, did you use the suggested flags on the install guide page?
reply