although whether humanity dies before the cat is an open question
The issue I see is the personification, some people give vehicles names, and that's kinda ok because they usually don't talk back.
I think like every technological leap people will learn to deal with LLMs, we have words like "hallucination" which really is the non personified version of lying. The next few years are going to be wild for sure.
Would you do that for drill, too?
"I dunno, the drill told me to screw the wrong way round" sounds pretty stupid, yet for AI/LLM or more intelligent tools it suddenly is okay?
And the absolution of human responsibilities for their actions is exactly why AI should not be used in wars. If there is no consequences to killing, then you are effectively legalizing killing without consequence or without the rule of law.