Yeah, I deal with multiple Go projects that take a couple minutes to link the final binary, much less build all the intermediates.

Compilation speed depends on what you do with a language. "Fast" is not an absolute, and for most people it depends heavily on community habits. Rust habits tend to favor extreme optimizability and/or extreme compile-time guarantees, and that's obviously going to be slower than simpler code.