Low quality "spam" tactics still reel in enough fish to be monetarily worth the "backlash" from customers that find it distasteful and or start to lose trust in the company. The "We promise we don't spam people" metrics and "Consumer distrust" metrics don't talk enough to the "revenue" metrics, but especially have very different cycles: big customer satisfaction metrics like J. D. Power are big annual things, not quarterly reports like earnings. In my experience things like "how often are we calling the same disinterested people to the point where it starts to feel like very personal spam" metrics don't ever really get prioritized in internal reports unless there's enforcement from Legal departments, and even then Legal departments can't "upset the bottom line" and only care about such compliance when it becomes News and/or Lawsuits, both of which don't even merit even an annual review cycle. (In fact, the modern class action lawsuit pretty effectively prevents that feedback mechanism from cycling, because generally the terms of agreement in a settled class action lawsuit is that the class is no longer allowed to sue again for the same problem, even if the same problem keeps happening and is never actually fixed.)
Unless quarterly earnings reports need to include things like client satisfaction and spam tracking, the only metric management will continue to care about, because it also is the only metric shareholders claim to ever care about, is the "bottom line", no matter how ugly the sausage is made to bump it from quarter to quarter.
So any big AI initiative they are apart of had succeeded before it even starts!
Especially in non-tech/non-product companies right now. Look at the age profile of owners and CEOs. Most of them were around for the dotcom era, and eventually they saw that companies that ignored the internet or didn't adapt where completely left behind.
There's a real fear among the C-suite/management that this is the same thing all over again, and if they are not fast enough on the adoption they will lose their business.
To them, AI is an existential crisis.
VS. the inadequacy of ai systems (nondeterministic output, no reference with reality, unknown signal to noise ratio, low effort etc).
But their bonus depends on driving down costs in their company.
I think this leads to a problem because leadership might see metrics showing that the AI service agent successfully helped with 80% of the questions it is asked, but they don’t realize how damaging that 20% is. Over time, more and more of your customer base is going to hit that 20%, so eventually everyone is pissed off at you.
It would be like if 1 in 100 Cheerios in a box were made of poop. It doesn’t matter that most of them are fine, people are going to remember that one cheerio more than the 99 others.
What's more interesting is the cases where it isn't. Those prove that the idea can be good, but it's obviously a lot more work than "have an LLM answer customer questions".
I had to call Geico's claims department yesterday and their AI customer service agent had a surfer bro accent, said "no worries" and "hang tight" and "I gotchu" while trying to follow-up on my claim which made it even more infuriating to interact with. Like... stop with the affectations and fake trying to be hip just be professional and succinct please!
I was just trying to schedule my daughter's dentist appointment and had to spend 10 minutes talking to an AI when I could have found a time that worked in 30 seconds with a human. And at the end of the whole process, she got my daughter's name wrong. It was demoralizing.
I realize the decision makers have been prioritizing the opposite. Making calls take as long as possible but I have no idea what is incentivizing them to do that.
I have no idea how to go about implementing such a thing, but it would be cool if someone picked up the idea and ran with it.
Have you seen "Severance"? It's a wonderful show that shows us a sick truth:
Many people are a different person at work. That different person is devoid of morals & ethics. They are machines intent on meeting metric targets. Nothing else, not managers, not employees, not people, just machines hitting targets they didn't decide on.
How could we possibly expect more?
Of course nothing beats a human with real agency at the company but like, these modern agents could be 100x better than what airlines and internet service providers currently have.