Nice, excited to see that. This is basically deeper pipelining, IIUC. Nick (the author of the blog) implemented the current version of this "emit metadata sooner," optimization and already experimented with pushing it even sooner a while back.
replyWe have discussed this briefly on last All Hands, the problem will probably be dealing with compiler errors, when they happen in builds that already emitted metadata. I believe this was the reason why it wasn't merged in the first place.
Yes I think I read a post of his from like 2019 were something was tried like this before. My guess is that this will be an interesting and hopefully informative PR for you all to see, but will likely not be merged as is. I had to add the ability for rustc to pause after early meta data emission and that was hacked together I feel, not well designed per se.
replyI view this as speculative execution/compilation and just throw out any errors from a crate that depended on another crate that ultimately errors out. It has the same outcome (same errors are printed in both cases) you just have a chance of having totally wasted some cpu time (that was otherwise just sitting around though).
Sounds lovely, give it a try on the codex-rs which seems to be getting close to using 2K crates in their repository any time now. Takes like 30 minutes to compile on a my beefy machine and it's a goddamn TUI, not sure what's going on, and not too interested in diving into that beast either.
replyThis thing is a monster, blows out the disk space on a Claude code web session. Will run it on my test server later and let you know.
replyNow I want to test it too! Need any testers with Threadripper systems to really see what it can do? ;) Email in profile
replyI will email you when I have something up for sure! First cargo check has it at about 28% reduced on codex rs, from 383 seconds down to 277 seconds on a 16 core machine. No -Zthreads yet though, and that typically has a similar effect and reduces the speed up the early meta data gives.
reply