Wait, Rust can already communicate using the C ABI. In fact, it offers exactly the same capabilities as C++ in this regard (dynamic linking).
I was addressing this portion of your comment: "C's ABI and dynamic linking are the thing that enables the software to get huge". If the C ABI is what enables software to get huge then Rust is already there.
There is a second claim in your comment about a "safe ABI", but that is something that neither C or C++ offers right now.
> There is a second claim in your comment about a "safe ABI", but that is something that neither C or C++ offers right now.
Of course C and C++ are no safer in this regard. (Well, with Fil-C they are safer, but like whatever.)
But that misses the point, which is that:
- It would be a big deal if Rust did have a safe dynamic linking ABI. Someone should do it. That's the main point I'm making. I don't think deflecting by saying "but C is no safer" is super interesting.
- So long as this problem isn't fixed, the upside of using Rust to replace a lot of the load bearing stuff in an OS is much lower than it should be to justify the effort. This point is debatable for sure, but your arguments don't address it.
I think we all agree that it would be a huge deal.
> - So long as this problem isn't fixed, the upside of using Rust to replace a lot of the load bearing stuff in an OS is much lower than it should be to justify the effort. This point is debatable for sure, but your arguments don't address it.
As you point out, this is the debatable part, and I'm not sure I get your justification here.
> It can't be that replacing 20 C/C++ shared objects with 20 Rust shared objects results in 20 copies of the Rust standard library and other dependencies that those Rust libraries pull in. But, today, that is what happens. For some situations, this is too much of a memory usage regression to be tolerable.
If memory was cheap, then maybe you could say, "who cares".
Unfortunately memory isn't cheap these days
This isn’t that kind of duplication.
E.g. the kernel wouldn't really benefit from a "safe ABI" because users calling into the kernel need to be considered malicious by default.
I think you're moving the goalposts significantly here.
Rust-to-rust code should be able to be dynamically linked with an ABI that has better safety guarantees than the C ABI. That’s the point. You can’t even express an Option<T> via the C ABI, let alone the myriad of other things rust has that are put together to make it a safe language.
You can look to Swift for prior art on how this can be done: https://faultlore.com/blah/swift-abi/
It would be very hard to accomplish. Apple was extremely motivated to make Swift have a resilient/stable ABI, because they wanted to author system frameworks in swift and have third parties use them in swift code (including globally updating said frameworks without any apps needing to recompile.) They wanted these frameworks to feel like idiomatic swift code too, not just be a bunch of pointers and manual allocation. There’s a good argument that (1) Rust doesn’t consider this an important enough feature and (2) they don’t have enough resources to accomplish it even if they did. But if you could wave a magic wand and make it “done”, it would be huge for rust adoption.
Thank you :-)
> It would be very hard to accomplish.
Yeah it's a super hard problem especially when you provide safety using the type system!
The work the Swift team did here is hella impressive.
> But if you could wave a magic wand and make it “done”, it would be huge for rust adoption.
Yeah!
Fil-C solves it. I think Swift solves it, too.
So it's solvable.
No fundamental reason, that I know of, why Rust or any other safe language can't also have some kind of story here.
> I think you're moving the goalposts significantly here.
No. I'm describing a problem worth solving.
Also, I think a major chasm for Rust to cross is how defensive the community gets. It's important to talk about problems so that the problems can be solved. That's how stuff gets better.
You cannot change anything that would affect the class layout of something in the STL. For templated functions where the implementation is in the header, ODR means you can't add optimizations later on.
Maybe this was OK in the 90s when companies deleted the source code and laid off the programmers once the software was done, but it's not a feature Rust should ever support or guarantee.
The "stable ABI" is C functions and nothing else for a very good reason.
In lots of domains, having a language that doesn't change very much, or that only changes very carefully with backcompat being taken super seriously, is more important than the memory safety guarantees Rust offers.
As a C++ developer, I regularly deal with people that think creating a compiled object file and throwing away the source code is acceptable, or decide to hide source code for "security" while distributing object files. This makes my life hell.
Rust preventing this makes my life so much better.
I mean yeah that's bad.
> Rust preventing this makes my life so much better.
I'm talking about a different issue, which is: how do you create software that's in the billions of lines of code in scale. That's the scale of desktop OSes. Probably also the scale of some other things too.
At that scale, you can't just give everyone the source and tell them to do a world compile. Stable ABIs fix that. Also, you can't coordinate between all of the people involved other than via stable ABIs. So stable ABIs save both individual build time and reduce cognitive load.
This is true even and especially if everyone has access to everyone else's source code
Rust supports ABI compatibility if everyone is on the same compiler version.
That means you can have a distributed caching architecture for your billion line monorepo where everyone can compile world at all times because they share artifacts. Google pioneered this for C++ and doesn't need to care about ABI as a result.
What Rust does not support is a team deciding they don't want to upgrade their toolchains and still interoperate with those that do. Or random copy and pasting of `.so` files you don't know the provenance of. Everyone must be in sync.
In my opinion, this is a reasonable constraint. It allows Rust to swap out HashMap implementations. In contrast, C++ map types are terrible for performance because they cannot be updated for stability reasons.
Am I wrong?
Firstly, of course you could.
Secondly, you don't even need to, as NixOS shows.
In some sense, the chasm I'm describing hasn't been crossed by C++ yet
Those folks think it doesn't.
I don't know that.
Here's what I know: the most successful OSes have stable OS ABIs. And their market share is positively correlated with the stability of their ABIs.
Most widely used: Windows, which has a famously stable OS ABI. (If you wanted to be contrarian you could say that it doesn't because the kernel ABI is not stable, but that misses the point - on Windows you program against userland ABIs provided by DLLs, which are remarkably stable.)
Second place: macOS, which maintains ABI stability with some sunsetting of old CPU targets. But release to release the ABI provides solid stability at the framework level, and used to also provide stability at the kernel ABI level (not sure if that's still true - but see above, the important thing is userland framework ABI stability at the end of the day).
Third place: Linux, which maintains excellent kernel ABI stability. Linux has the stablest kernel ABI right now AFAIK. And in userland, glibc has been investing heavily in ABI stability; it's stable enough now that in practice you could ship a binary that dynlinks to glibc and expect it to work on many different Linuxes today and in the future.
So it would seem that OS ABIs are stable in those OSes that are successful.
Leaving aside the platforms it no longer supports.
So there are some changes to account for depending on the deployment scenario.
- the same entity has access to the source of both the library and the main app
- library and main app share the same build tooling
And even if that’s the case, you have the problem of end users accidentally using different versions of the main app and the library and getting unexpected UB.
It's also as bad as C.
I'm saying that the chasm to cross is a safe ABI.
1) It can't be that replacing 20 C/C++ shared objects with 20 Rust shared objects results in 20 copies of the Rust standard library and other dependencies that those Rust libraries pull in. But, today, that is what happens. For some situations, this is too much of a memory usage regression to be tolerable.
2) If you really have 20 libraries calling into one another using C ABI, then you end up with manual memory management and manual buffer offset management everywhere even if you rewrite the innards in Rust. So long as Rust doesn't have a safe ABI, the upside of a Rust rewrite might be too low in terms of safety/security gained to be worth doing
Also unsafe rust has always on strict-aliasing, which makes writing code difficult unless you do it in certain ways.
Having glue libraries like pyo3 makes it good in rust. But that introduces bloat and other issues. This has been the biggest issue I had with rust, it is too hard to write something so you use a dependency. And before you know it, you are bloating out of control
They have been working around it with DLLs, and COM/WinRT, but still the tooling isn't ideal.
You can also access .NET/C# objects/interfaces via COM. It has an interface to allow you to get the type metadata but that isn't necessary. This makes it possible to e.g. get the C#/.NET exception stack trace from a C/C++ application.