E.g. look at a Python list. Is it safe? In Python sure, but that's abstracting a C implementation which definitely isn't safe.
If you look at Rust's std::Vec you'll find a very similar story - safe interface over an unsafe implementation.
It isn't as binary as you think.
It's true that safe wrappers around unsafe code sometimes have bugs in them, but it's orders of magnitude easier to get the abstraction right once than to use unsafe correctly in many places sprawled across a large codebase.