https://corrode.dev/blog/tips-for-faster-rust-compile-times/...
FWIW - last stage where the binary is produced takes the longest and is single threaded and that's the largest difference between release and debug.
At a quick glance, most benchmarks show only 50% to 60% single threaded improvement between M1 and M4.
That wouldn't explain compilation times going from 6 minutes to 2 minutes.