If you were to implement a Rust GC, then Rust can guarantee that references haven't escaped the current thread, which means there is no stop the world pause anymore, only the current thread gets paused, which is acceptable.
Until you need a library that was written with the assumption of using a garbage collector.