The biggest difference is the failure modes. If I'm not thinking about memory, my RSS is higher or a bit of extra CPU time goes to GC. Both of those are radically better than UAF or buffer overruns. Good trade IMO.
A GC is not the only fix for UAF and buffer overruns...
That's part of the reason why Python, go, Ruby, etc. are so popular.
There is no one right answer, it's very dependent on what's being built and where the ROI for the programming effort comes from.