I have also learned that rarely does anyone care if it’s any good, or means anything. This is generally true, but it’s especially true if you are going with the prevailing winds of whatever management fads are going on.
Like, right now, you can definitely get away with inflating the efficacy of “AI” any way you can, in almost any company. Nobody with any authority will call you on it.
Look at what management’s talking about and any pro-that numbers you come up with can be total gibberish, nobody minds. “Oh man, collecting good numbers for this and getting a baseline etc etc is practically impossible” ok so don’t and just use bad numbers that align with what management wants to do anyway. You’ll do great.
If the company were an airplane essentially upper management were flying it by instrument. It would've been a scandal if the metrics had serious issues.
Some of the metrics less directly tied to business stuff were a bit more 'creative' - as in I could justify why I did them that way, but still not 100% solid.
Stuff like optimizing data pipelines, where data scientist experiments which tended to take 1hr, now only took 10 mins.
I could say that data people were 6x as productive, but it's just as well possible they were just more careless with what they ran, but whatever, a white lie.
However saying that stuff takes 1/6th the time, when in fact it doesn't, is an absolute no go. Neither is not knowing why is there a run that took 500 hours or 5 seconds, both of which should be impossible.
Doing that stuff destroys the confidence in the rest of the data.