upvote
A RewriteInRustBench would be unironically useful at this point since all the main agents can write it reasonably well despite its relative scarcity in the input data.
reply
Rust is the best language for LLMs b/c it gives by far the best debug messages. Just tons of verifiable reward signal for post-training. Even the most rudimentary LLMs can school me on idiomatic Rust
reply
On the other hand, Rust's borrow checker is very picky, and even a frontier LLM still sometimes struggles to respond to roadblocks sensibly (refactoring so whatever it's trying to do can be done safely) rather than stupidly (introducing some horrible global arena thing so it can make the borrow checker go away). A lot depends on how good your instructions are, and how good the existing code is, since bad input begets bad output.
reply
I've (more or less; I've read quite a bit of the code) vibecoded several houndred thousand lines of Rust and I've not seen this happen a single time. It sounds like something it'd do when you ask it to "write a linked list while satisfying the borrow checker". Are you sure you haven't (possibly unknowingly) been giving it instructions which ended up luring it into doing these things?
reply
Evidence needed. I think for certain kinds of outcomes it has very strong advantages, but these advantages are not a given as 'best for LLMs' :)
reply
Not always. In my experience, if you're not working on a small, trivial codebase, LLMs will sometimes just create spaghetti unreadable, inefficient code to satisfy the constraints of the type system/borrow checker.
reply
Have each agent rewrite openssl in $lang and call it the RollYourOwnCrypto bench.
reply
I wouldn't be surprised if we're already at the point of more LLM-written Rust than hand-written. Models training off models
reply
All the main agents can write Jai code reasonably well despite being even more scarce in input data!
reply
Not many people can hold grudges as strong as principal engineers
reply
I wonder if this means Carbon is DOA.

I was excited to see what it would be. But I don't think I can argue that it makes as much sense anymore.

reply
Carbon was clearly DOA the moment it was announced, IMHO. It looked cool but it served none but Google, and now with LLMs you have a massive incentive not to use a niche or new language due to how better LLMs get the bigger the corpus is

The only somewhat realistic proposal in this space is Herb Sutter's cpp2, which is arguably a massive improvement and I'm puzzled why nobody in the standard thought to give it a spin, there's just to much cruft they'll never be able to get rid of unless they make an alternate yet backward compatible syntax with C++ that changes the defaults from "random 80s nonsense" to something better

reply
Cpp2 has been dead for a while afaik, while Carbon is still going.

I don't think Carbon is dead, it just all depends on how easy it actually is to rewrite "all of C++" in Rust. (The jury is still out on this one, but it's not looking good.)

reply
Version 0.0.0.0 after 4 years. Their goal of "full interop with C++ while being a completely new language without any of the flaws of C++" is plain absurd.

It's DOA because Google doesn't have any idea of what Carbon should be, and to be completely honest, at least 80% of what they currently use C++ for should be rewritten Go, you know, that language developed specifically because of the issues with C++ by teams within Google.

reply
Rewrite everything in Rust has been a meme for so long that to see it coming to pass is surreal.
reply
I wonder if there's people already whose full time job is maintaining/extending one of these auto-migrated codebases.

Imagine they aren't even familiar with rust but are deeply familiar with the product.

reply
What exactly makes Carbon absurd?
reply
There is 0 practicality in inventing an entirely new coding language that only one company uses, and you have to teach it to thousands of new engineers. Rust exists and fits the job totally fine and is used in more places and has actual support outside of a single entity (i.e you can actually hire people that feasibly know the language).

It was clearly done because some PL guys at google really wanted to make a new cool language and Google was the perfect place to incubate it without it getting axed. Probably got a couple of promos out of it too. This is clearly not the best use of time or money, but I guess if you're google you have so much of both it probably doesn't really make a dent, and you can keep a few very smart people happy with shiny new projects.

Also, LLMs being used for a large portion of coding nowadays sort of remove the need for these types of languages, IMO. They make less "silly" bugs (both logical and structural) that languages like this are meant to catch, and they are much better at languages that are better represented in the training corpus. This somewhat obviates the need for very niche "type/dummy-safe" languages like carbon (and even rust/zig, imo). So even if you did want to use Carbon, you'd likely have to bootstrap a decent amount of your own "good" carbon code to post train an LLM, and even then, it likely won't have that big of a gain vs just having an LLM write C++ or even Rust. If you are a company that still reviews code, you should just have an LLM code in a language most people can understand anyway to make verifiability tractable.

reply
Rust is not the end of history. One of the difficulties with the language lies exactly with porting existing code written in an OOP style to idiomatic Rust, as those codebases weren't written with ownership in mind.

Such rewrites will contain judicious uses of Cell, RefCell, unwrap() etc. which make for ugly code that's not exactly simple to understand and might even have some landmines (crashes).

Getting rid of these requires a subtantial amount of engineering effort, which I'm not sure how well these LLM manage.

Given the nigh-universal experience of LLMs producing an ungodly mess when left to their own devices, I have my concerns.

reply
> There is 0 practicality in inventing an entirely new coding language that only one company uses, and you have to teach it to thousands of new engineers

They did that for Go and it seems to have worked out for them though.

reply
Hmm, I don't disagree with you that LLM's remove the need for type-safe languages, but as the blog mentioned, Google is porting their C++/C code to rust. Does this mean the port is waste of time and that they should just rely on the LLM's to catch memory errors?
reply
I mean Rust definitely has a better tradeoff than Carbon in this case, re readability/verifiability by a person (and sufficiently good internet training data).

I personally think that you _could_ use an LLM to catch these types of boundary case errors without having to port the _entire_ C++ codebase to Rust, but maybe pre-emptively porting to Rust now can catch some of these cases for cheaper than doing a full LLM sweep. Also more cynically, its a good benchmark lol.

I guess if you really believe in curve of LLM capabilities you should just use a language that has the best performance, safety, flexibility, and extensibility, since in the limit few/no people will actually read the code anyway. I think this ends up being Rust.

reply
I'm not an expert on this. But isn't it the case that C++ code could have errors that span the entire codebase, like a setup in file A triggered by a bug in file B which is immensely far away on the import graph? A classic would be a use-after-free. To me that's the thing that Rust can help with, even if silly bugs aren't being written by AI.

The other thing is just that rewriting some old human-written codebase in Rust probably immediately catches many bugs. It would be hard to prompt the AI to properly scan for such bugs itself, they're lazy when working in that modality.

reply
Didn’t FaceBook fork php into another language?

I’m not entirely sure Google should have both Go and Carbon but when you have billions in server costs it makes sense to do extreme stuff for even basis points of performance. I’m still surprised at how much java there is.

reply
deleted
reply
I’m not op. But I think it’s not Carbon itself that is absurd.

It’s absurd to think that Carbon is the solution to memory safety when rust exists and Carbon’s memory safety story is basically “TBD”.

reply
I wouldn't call it absurd, but very questionable at least. Most companies are not going to even consider throwing money at this adventure.
reply
The fact that it will never exist.
reply
rust's existence?
reply
Heh, I use my clankers to rewrite Rust in C
reply