If you don't need to rebuild anything, bazel can fetch only the final artifact (not the intermediates) from the remote cache.

Also, if you have persistent CI workers with a persistent bazel instance, you save on some network roundtrips, but that's obviously harder to set up and make bulletproof.