Not disagreeing with the rest of your comment, but regarding this tangent: haven't there been a ton of (say) "AI" companies that quietly used humans in the background and gave the impression AI was doing the work, because their vision was that that was the future?
Not that I think that would excuse this case in any way regardless - just mentioning because of the drawn comparison.
These stories were popular during the period when people didn’t understand how AI worked or what was possible. It was feasible to produce fake demos that relied on humans doing the work because few people understood how AI worked. If your demo took an input and produced an output 1 to 12 hours later with variable delay nobody questioned it because they didn’t know what AI looked like. They thought maybe that’s how long the computers had to think or that there was a single computer working through a backlog.
Now everyone has used ChatGPT and understands that actual AI doesn’t work like that, and if the task actually took 12 hours of crunching on tokens the bill would be an eye watering large number. So you can’t get away with faking your AI responses with a few humans doing the work.
There were also follow on stories where writers got confused about hybrid systems that had humans doing review and RLHF, thinking that any involvement of a human in the loop was equivalent of faking AI by having humans do the work. Most of the stories I saw toward the end of this trend were this style.
They’re also unfamiliar with FDA-regulated spaces where it’s not unheard of for VPs and C-suite types to receive fines and jail time and bans from the industry.