The public counterpart is tricky to name but I think attaching "public" to it makes the intended usage plenty clear. There isn't really a physical counterpart unless you consider maybe those machines that check for counterfeit cash but even that's not a great fit because the pubkey is simultaneously analogous to a lock box.
It is true that publishing your private key is bad but you'd hope the name makes that pretty clear. Despite the way I remember (U2's "The Fly" lyrics, "A secret is something that you tell one other person, so I'm telling you, child") people generally do not understand that the whole point of secrets is that at least two parties know, which means you might always be betrayed by somebody you think is keeping your secret. For a private key it's easy, don't tell anybody, nobody knows, you can't be betrayed, done.
For example Hacker News learns my password to this web site every single time I sign in because that's just a secret. We've known how to do better for decades but only a handful of systems I use (e.g. Google) do so and all of them have a "traditional" password option which is like discovering your aeroplane still has a smoking section in 2026.
I don’t think there’s any process or entity in the physical world that is reasonably familiar to most people that is even remotely suitable as an analogy to public key cryptography.
These are not analogs to things ordinary people had already seen. For Computation we just got used to it being everywhere and so we don't need to explain it so much.