upvote
I would replace the word scientists with engineers in that quote. People often conflate the two, but in my experience, scientists tend to be more cautious and there are built in checks and balances in the process (however flawed).

Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.

reply
This sounds like you don’t have much exposure to actual professional engineering disciplines. I’m sure civil, electrical, structural and mechanical PEs would be quite surprised to hear there are no guardrails on their professions.
reply
Engineers versus "engineers".

I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)

reply
I prefer the term "software developer" and that's what I use when I don't need the prestige of the term "software engineer". It's disadvantageous for organizations to do that with actual job titles, though.

Absent US government intervention to codify the term "engineer", probably the only way out of the "engineer" trap is through further title inflation, where the developers all become "vice presidents". :)

reply
People meeting the definition of Software Engineer while having that title may be rare, but we certainly need more of them.
reply
Yeah, it's 100% the better term. We've got rules against using engineer here in Canada though several companies I've worked for have called me an engineer. Apparently Professional Engineers Ontario sometimes goes after people for calling themselves engineers but I've never heard of it actually happening, and I don't know that they have any real teeth given that the places I worked that called me an engineer were Canadian-owned. (In fact, the only place where they checked if I could use the title was the one multi-national. Go figure.)
reply
In my experience “engineers” and builders are often quite “conservative” and really don’t like pushing the envelope, and they often only do it under protest.

The most famous example may be the perpetual war between architects and engineers/builders.

reply
I actually originally wrote "technologists" but thought that the word sounded kind of odd. Now I realize it better captures what I was trying to convey.
reply
Researchers need to go wild and sometimes far off-the-rails to increase the odds of coming up with something that is both new, and potentially popular enough, if they want the option to attract marketers who can only thrive on mass-consumption.

With luck, one out of 100 inventions will show promise on those points.

There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.

OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.

It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.

Perhaps popularity alone is not always the best measure of progress.

reply
I worry that companies like Anthropic, which are moving toward RSI, may prioritize speed over the timely identification of irreversible risks.
reply
We need to force the executives and engineers working at Anthropic and other AI companies to read this encyclical and pass an exam demonstrating that they read it.
reply
Every time I prioritize speed over risks, I, too, end up with repetitive strain injuries.
reply
Did you mean AGI? I'd think whatever else they are doing, LLMs are reducing RSI...
reply
Recursive self-improvement
reply
Maybe. I am not convinced I type fewer keystrokes being a Markdown Monkey compared to writing code largely via autocomplete. Fewer Tab keystrokes, for sure.
reply
I was mostly making a joke.
reply
[dead]
reply