upvote
reminds me of a startup I was in, where one of the devs accepted a promotion to head of engineering on the condition that the CEO would not override him when he said a release was too buggy to go out to customers.

the CEO agreed because (I imagine) he really needed someone to take that position. and as everyone should probably have seen coming, the first time he really wanted to send out a release bugs and all, he called the guy into a room and pretty much browbeat him for an hour about how making the promosed release date was more important than making a good release, until he said "fine but I'm not responsible if it breaks".

the CEO held this up as an example of how he had kept his word not to send the release out without the guy agreeing to it. that startup, needless to say, is long dead.

reply
I find this anecdote fascinating because, while I identify completely with the new head of engineering, his requirement really only makes sense if you interpret it as a commitment to build at a certain level of quality, business be damned. Which is fine, but then he folds at the first test of this principle! Didn't even let the business sweat for a day!

This could be a lesson for us engineers to be a little less glib and a little more introspective, especially as you gain more power and influence in an org.

reply
I think in this case it was more a plea to have his judgement valued when he said "dude, if the qa team has found this many bugs this is only going to harm our reputation with the client, and we are going to be the ones doing emergency bugfixes at midnight". but also the company had a pretty toxic culture where the CEO knew how to manipulate people to get his way. he definitely managed to guilt trip me into working overtime when he told clients we already had features that we didn't in order to close a sale, basically "we've worked so hard already and we really need to close this sale, can't you just work up a basic version of it over the weekend?"
reply
I wonder if the better approach would have been to lead engineering with a similar reliance on bending rules and "what you can get away with". This is where multiplying estimates by PI comes in, I think. And possibly some creative division of responsibilities to make it harder for a CEO to officially bully engineering. Totally different job than most engineers would (or could) want to perform- the guy probably wouldn't have taken the promotion in this case!
reply
Doubtful. I suspect the CEO was the one doing the estimating, so any number you give him is ignored in favour of what he provides instead.
reply
you are indeed correct
reply
> What do you think is more likely: this or the other way around: getting the "ethics team" to align with investor goals? Hint: Where does the money come from?

From the (ethics) team obviously, because the investors just have goals and aren't a team!

Big irony marker of course. But remarkably, in human history, this line of thinking would not be unheard of. Of course, I don't think it easily adapts to companies, so I don't really disagree with you.

Remember that MTV show offering people like 5 grand or something to lick an elevator handway in front of a camera?

Well, if someone has a useful idea in this world, and want to build a company from it, the MTV "lick-the-stairway" offers will be the largest and first hurdle before anything else happens. People will simply offer to buy you out, and that's not limited to founders and creatives.

That's how our system works.

reply
I feel I missed some logical step in-between "corruptive power of money" and the elevator/stairway railing/banister thing.

> People will simply offer to buy you out, and that's not limited to founders and creatives.

A less-obvious failure case would be bankrupcy court, where all sorts of ethical promises and even explicit contracts may be voided in pursuit of recovering a buck for creditors.

reply
Yeah, I've never found the "maximizing shareholder profit" imperative to be compelling as something that should be done, but as a predictor of how companies will act, it's not something I ever feel safe betting against.
reply
One subtlety that is worth considering is that what a company is inherently set up to do is maximize shareholder value. Value can be quite complex, especially while the number of shareholders is small. Value can include the legacy or reputation of the shareholders. It can include the ethics of the shareholders. It can include completely different goals, like the famous "make humans a multi-planetary species".

Treating value as purely monetary profit is a perversion of that. To a certain degree it's inevitable as the number of shareholders grows, so seeing it in public companies isn't surprising. But even then, if a company focused on creating equipment for sustainable, pesticide free farming were to pivot into making Hellfire missiles that might be insanely profitable while still destroying a large part of what existing shareholders value about the company

reply
It's funny, but Scientologists were innovators on this front. They use the word "ethics" a lot, but they've specifically redefined it to mean productivity at work (people who aren't productive are stealing a living.)

Being "downstat" (iirc) is when you're not productive in comparison to your coworkers, which makes you "outethics" and a "potential trouble source (PTS)." If you're outethics, then you get called in for "auditing" which is when they interrogate you with a lie detector to find out if you're associating with "suppressive people (SP)" (who are people who are causing you to be downstat because they despise human happiness.) If those people can't be found, then the problem is obviously in your "withholds" (again, iirc) which are your secret deep-down desires to destroy the organization that you may not even be aware of. You see, your "reactive mind" is raging at being forced to be "ethical." The conclusion is that either you find the SP and "disconnect" from them, you discover the nature of your withhold and admit that you were plotting against the organization and why, or you're the SP and you get declared and ejected from the organization.

Welcome to "ethics." The original AI alignment scholars.

The specific, named people who run these organizations are moral black holes. Anybody that they're hiring for "ethics" they're hiring to define an ethics for their own benefit.

reply
Great point.

The EA group have a ton of parallels to the Scientologists, so all of this is not surprising.

reply
Microsoft Tay has entered the chat.

I'm kind of surprised by the cynicism. There are real problems in AI ethics that contribute to model training. Someone has to own those problems. How that team is incentivized is outside the scope of my comment.

reply
The cynicism, I believe, is mostly directed at corporations themselves. Corporations, especially AI companies, are anything but inherently ethical. Their very existence has depended heavily on the use of data obtained without explicit permission, including copyrighted material. One can therefore reasonably suspect that the main purpose of having ethics teams in these companies is to boost their credibility, provide a layer of plausible accountability, or create a convenient scapegoat that can be sacrificed when necessary.
reply