upvote
For the curious, this is technically possible in Rust using a dynamically sized type [1], but in practice is difficult and doesn't really play nice with the rest of the language. The nomicon entry concludes with "Yes, custom DSTs are a largely half-baked feature for now." [2]

[1] https://doc.rust-lang.org/reference/dynamically-sized-types....

[2] https://doc.rust-lang.org/nomicon/exotic-sizes.html

reply
> putting the record data right after the CacheEntry members

I assumed they couldn't do that because they're using it with some kind of generic HashMap<K, V>. In that situation, can "V" be dynamically sized?

A dynamically sized "V" would mean you can't have an array of them, which might preclude some hash map implementations.

reply
> All type parameters have an implicit bound of Sized. The special syntax ?Sized can be used to remove this bound if it’s not appropriate.

, which HashMap does not do, i.e. the keys and values have to have a statically known size.

reply
deleted
reply
Unfortunately, Rust is not a good choice for this kind of tricks. This is where Zig shines. In Rust, you can’t even use proper arenas, which can help a ton with allocations.

Cloudflare started to pick Zig recently, for projects, that have memory constraints.

reply
> In Rust, you can’t even use proper arenas

You definitely can and this is done a lot. What you might mean is that you can't use standard library's collections with them (this is getting stabilized soon!) and have to use third-party, but that is a different thing than "can't use arenas".

> Rust is not a good choice for this kind of tricks.

Rust can do those tricks, but it's true that it is hard than in C or Zig. That said there are often crates to help.

reply
Stabilized soon, really? They did not stabilize it after 10 years and were thinking about different approach. I thought it’s dead.
reply
Yes, really. The design has been decided upon ( https://hackmd.io/nNHdKkp1TTK7jat0I-ABqA ) and the implementation has been updated to match ( https://github.com/rust-lang/rust/pull/157428 ). The stabilization PR is just waiting on final approval by the relevant team members, with no remaining concerns currently listed: https://github.com/rust-lang/rust/pull/156882#issuecomment-5...
reply
I'd like to know why I can't use arenas in rust? Especially considering that I have used them before in rust.
reply
You can’t allocate collections without nightly or without reimplementing them in the library. Every implementation uses it’s own set of trade offs to provide safety in unsafe implementation.
reply
Rust supports arenas just fine ( https://crates.io/crates/bumpalo ), and if you mean the support for using custom allocators in the standard library collections, that's as stable as Zig is.
reply
System programming always matters. Things are cheap until they aren't one day.
reply
things are cheap until you reach a scale.
reply
Things are cheap until they are someone else’s problem, I say!
reply
deleted
reply
deleted
reply
Depends on how the CacheEntry is stored, it's probably stored in a slice of &[CacheEntry] which precludes storing the record data alongside it as the size of each entry must be fixed.
reply
This is where hand-rolled intrusive data structures, as are traditional in C, really shine.
reply
I wish more programming languages implemented record types as seen in databases, where dynamically sized fields are packed into a contiguous area of memory.

The CloudFlare manually implemented a clumsy version of this.

Wouldn’t it be nice for the compiler to manage this for you in the same way that your database engine does when it saves a “row”?

reply
> dynamically sized fields are packed into a contiguous area of memory

Are you able to explain this? Do you mean an N sized array where each entry is either a value or a pointer to a value where the 'pointed-to' values are after the end of the array?

I'm trying to underatnd how you'd do this without having to parse M-1 elements to get the Mth entry if you did a [{size0, value0}, ....., {sizeN, valueN}] arrangement

reply
I think they mean the cache entry is a collection of dynamically sized fields. It would be nicer to store that as a single contiguous allocation, rather than a bunch of pointers to individually allocated dynamically sized items. At least in this case, it might.

In a row oriented database, you get a contiguous spot for the whole row even when there are multiple variable width fields.

reply
less ergonomic, but still totally doable
reply