It's not unknown in human warfare either:
https://en.wikipedia.org/wiki/Human_wave_attack#Use
There's a similar human need to protect what is precious. Defending your family, defending your motherland. Cities being seiged by a superior opponent don't just abandon the city in cowardice, because they know the opponent will just gain ground and still come after them. They don't want to, they'd rather you stopped, but they have to defend their territories, even if it means their annihilation. If you win, everyone gets killed, ravaged, enslaved, ... If they only have sticks, and you have missiles and jets... they have to use sticks. Can you imagine how they feel?
But yes, you're probably thinking of RPGs where the enemies are preprogrammed with the same hostility no matter what the circumstances, even if you're a God-killer and they're defending 5 coins and half-eaten sandwich in their den.
The original Gothic for example. When you were high level, other low level NPC would rather run.
I also thought hard about the concept, how to make FPS games still fun, but a bit more realistic. The thing is, in most settings this means reducing lots of enemies - as realistic would be, once you start shooting, they all come for you. Not 3. And then maybe another 2. And so on.. and then you would not have a chance, unless you get special powers (or quick save and quick load part of the mechanics)
I see somebody in another comment complaining that enemies who get frightened rob the player of the fun of battles. So it depends what it's all about.
Chasing enemies is much more annoying than them coming to you, so that would be a punishment to the player.
Players don't like when you punish them in that way, they want to kill the monsters they don't want an upgrade that makes you more powerful make it harder to kill monsters since now they start running.
It seems I've failed to express myself clearly. The idea was that at some point, "low-level" adversaries simply stand no chance against a "high-level" PC, which should be obvious to both sides -- so acting accordingly on both sides would make sense without taking the fun out of the game, because -- and hear me out -- at that point in the game, chasing the low-level minions should be the last mechanic the player is forced to endure. When you are going for a dragon, you should not be forced to stomp your way past overly self-confident "newts" or mow down swarms of goblin youngsters…
Naturally, if the PC chooses to chase minions fleeing in terror after they took out the most courageous (or silly) third of their clan, that should be an option… and arguably it could even bring some satisfaction after the PC's low-level struggles, perhaps. But should this be the main mechanic? Definitely not -- at least not in the kind of game my thought experiment addressed.
But when humanoid enemies behave in plainly stupid ways it's a real immersion-breaker for me. I've been gaming a looooong time, so I'm quite adept at the necessary mental gymnastics to enjoy stuff anyway... but... still... games could be better here