upvote
Lack of TCO is also a common footgun for Scheme programmers using Common Lisp.
reply
This footgun is the reason I'm so enthusiastic about the Rust `become` keyword.

This proposal would give Rust a specific keyword which says that you intend TCO and so two things happen: 1. The compiler goes to more length to deliver TCO even where it wouldn't "just work" and 2. If it cannot deliver TCO your code doesn't compile, because you asked for TCO.

reply
Sounds similar to @tailrec in Scala

I personally use the phrase "tail call elimination" when it's a requirement that can be relied on; and "tail call optimisation" when it might be implementation-dependent, context-dependent, limited (e.g. to immediate self-calls), etc.

reply
I am definitely not a Scala expert.

As I wrote in a sibling comment, the key benefit here is the extra work from the compiler to deliver what you wanted, on top of the diagnostic if it can't.

I don't know if Scala has the problem that `become` addresses (C++ calls this RAII, but I have no idea what Scala would call it if they have the same idea)

However in my brief attempt to validate what Scala does do here, I found discussion of "always" optimising to a loop which is a bad sign. Tail recursion is an elegant way to write some loops but that's not the only thing it's useful for, and it seems as though Scala just doesn't care about other cases, at least for @tailrec

One thing you want TCO for in a language like Rust with lots of monomorphisation is to avoid function call overhead for the deliberately out-of-line slow path in some code. So in this case there was never an implied loop and we're not averting a stack overflow, we wanted to do a single instruction pointer change instead of an expensive function call wrapper. Seems like @tailrec isn't for that.

reply
I jsut did some digging and it seems you're right, it's only for methods which call themselves (which indeed get compiled into a loop, as an entirely local transformation). So not hugely useful.

Apologies, I've not written Scala for many years; I just recalled that there was a way to annotate tail calls which the compiler checks. I didn't realise it was so limited!

reply
Scala is in the way to get capture checking for effects, which will allow to do RAII like stuff, or borrow checker like stuff for that matter.
reply
Sounds like clang::must_tail?
reply
I am not a Clang expert, but first, obviously that's a C++ attribute and so while Clang can decide what it means in Clang in the programming language itself it has no semantic weight because the ISO document says attributes are always ignorable.

Secondly however in these languages you often won't naively get TCO because you have at least one local variable which C++ would say has a "non-trivial destructor" or Rust would say "implements Drop". These both mean that naively the "tail call" wasn't actually the last thing to happen, the destructor / Drop::drop happen at the end of the function, after the tail call.

The proposed become keyword tries to core::mem::drop any such variables, if it succeeds now that tail call is last and we can do TCO, if it fails [e.g. because the variables it wants to drop are needed for the tail call] we can diagnose the problem. I believe the Clang attribute doesn't have this behaviour.

reply
Clang tail-calls aren't guaranteed to work with all C++ code. If you have a non-trivial constructor, as you mention, it will tell you this and fail instead of silently letting you believe you have tail-calls when you don't.
reply
There are far more cases. Some ABIs use callee-saved registers for parameter-passing under certain circumstances, for example. Usually, there are compatibility restrictions on the signatures of the current and tail-called functions beyond the return type, too.

This is different from Scheme or the MLs (there as a quality-of-implementation feature) where tail calls into arbitrary functions are expected not to lead to space leaks.

reply
Right--if the tailing isn't possible, for any of these varied reasons, it will become a compile error.
reply
I have to say that at least knowing if I didn't get what I wanted is most of the value for me.
reply
Reordering destructors is not safe in C++, as it's fairly common to rely on objects being destroyed in reverse order and doing stuff like

  A a;
  B b(&a);
In rust the borrow checker would guard against reordering such things, but a caveat is that there might be unsafe code relying on drop-order which the borrow checker would be oblivious to. There could also potentially be objects representing external resources like a temp file where dropping them out of order leads to issues.
reply
> In rust the borrow checker would guard against reordering such things

It doesn't even get that far: Rust guarantees that things drop in reverse order of declaration, full stop.

One interesting wrinkle here: for struct members, Rust does the opposite of what C++ does. We debated changing it to match, but

> there might be unsafe code relying on drop-order which the borrow checker would be oblivious to.

There was no super real compelling argument to choose one direction over the other in the abstract, and "be the same as C++" was not considered important enough to risk breaking unsafe code that relied on the (what was at the time) implementation defined behavior.

reply
> It doesn't even get that far: Rust guarantees that things drop in reverse order of declaration, full stop.

The drops happen (if implemented) in the same order, but in a different place, half the point of become is to put any needed drops first before the call, as otherwise it's not in tail position and we can't do the optimisation.

So the borrowck can become involved if our become foo(bar, &baz) borrows baz but baz's type impl Drop - the diagnostics aren't great today, but then the feature isn't finished so it's not a priority.

reply
I was talking about regular old today's Rust, not the specifics about become.
reply
That Rust was in fact always unsound if it would cause problems to core::mem::drop(a); and the `become` call just drops things so it's the same.

Safe-but-undesirable outcomes are acceptable. For example maybe our tail call ends up reverting a database transaction and we wish it were otherwise. But if the code did compile but wasn't memory safe as a result of this new drop then it was always unsound and shouldn't have existed.

Just as the guts of some STL classes are very complicated in order to deliver the promised exception safety promises, the guts of unsafe Rust code are often tricky for similar reasons, you are mandated to deliver safety, it's not up to you to say "That's stupid, don't do that" either ensure it won't compile or safely cope.

reply
Does 1 really happen? I would never trust a compiler where 1 was a possibility. If it can work it should.
reply
Only if they are using an insufficiently smart compiler. SBCL handles TCO just fine, as do a number of other implementations, see : https://0branch.com/notes/tco-cl.html
reply
Even SBCL doesn't do TCO at all times. Compiling at (debug 3) means no TCO.

Another related footgun is deep recursion of other kinds, for example when recursively traversing down lists. For long lists it's easy to exceed the stack size limit. The common idiom is to recur on list elements, but iterate or map to go along a list.

reply
> Even SBCL doesn't do TCO at all times. Compiling at (debug 3) means no TCO.

Presumably one intends to debug the code, when setting (debug 3). Then it'll be helpful to see the stack, no?

> Another related footgun is deep recursion of other kinds, for example when recursively traversing down lists. For long lists it's easy to exceed the stack size limit. The common idiom is to recur on list elements, but iterate or map to go along a list.

Not going to argue with seasoned lispers here, but IMHO recursive code makes most sense when accessing recursive data structures.

reply
One place where this shows up is in parse trees. The grammar for a list of things may involve productions that look like list constructors. This, directly translated into a data structure, would give a very long chain of parse tree nodes dangling off to the right. It's a recursive data structure, but a very deep one for large lists, and traversing it recursively can use a lot of stack.

This can also be seen as an argument against building parse trees that way. Instead, have a node with an unbounded number of children, the elements of the list.

reply
> Presumably one intends to debug the code, when setting (debug 3). Then it'll be helpful to see the stack, no?

You don't necessarily need to give up TCO to do that though. You just do some bookkeeping and synthesize virtual stack frames. DWARF has native facilities to handle this.

CL goes the route it does mostly out of history, which includes the fact it has its own debugging ecosystem, more than any fundamental technical reason. There are technical hurdles with doing this in an image-based dynamic compilation model, but it's very far from intractable. Especially if you just do what GHC did and add a DWARF workflow. Most CL users wouldn't ever touch it though, because that's a drastically different debugging model that costs them a lot of ergonomic power, which may even be the reason they're working in CL to begin with.

reply
If it's not encoded in the language's specification, it's not a feature of the language but just an optimization. You can't rely on optimizations for correctness.
reply
in theory, no, in practice, yes.
reply
Mostly because they forget Scheme is one of the few languages where TCO is part of the language standard, making it a required feature for any compliant implementation.

This has always been an issue regarding TCO support across programming languages.

reply
Well, and also because of the "I've been told in Scheme you should do it this way, so by gum I'm going to do it this way!"
reply
Js really should have it. I think the shift in style from functional and manual prototype chains to Java classes is quite disappointing.
reply
Technically TCO is still in the spec, TC39 deadlocked over modifying it. TC39 really does not fill me with confidence in general.
reply
ES6 class syntax is still mostly just syntax sugar overtop prototypical inheritance.

JS _does_ still have TCO (called Proper Tail Calls), Safari's JavaScriptCore implements it, and is technically the only conforming interpreter.

reply
Yes but the syntax encourages patterns which would be uncommon in pre-ES6 JS.

I can’t rely on TCO if chromium doesn’t have it.

reply