"Software engineers shall act consistently with the public interest. In particular, software engineers shall, as appropriate:
1.01. Accept full responsibility for their own work.
1.02. Moderate the interests of the software engineer, the employer, the client and the users with the public good.
1.03. Approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish quality of life, diminish privacy or harm the environment. The ultimate effect of the work should be to the public good. ..."
From the IEEE, which also encompasses computer engineering, their first principle and its first few sub-items are:
"To uphold the highest standards of integrity, responsible behavior, and ethical conduct in professional activities.
1. to hold paramount the safety, health, and welfare of the public, to strive to comply with ethical design and sustainable development practices, to protect the privacy of others, and to disclose promptly factors that might endanger the public or the environment;
2. to improve the understanding by individuals and society of the capabilities and societal implications of conventional and emerging technologies, including intelligent systems; ..."
Of course people are going to ignore it if there's no force behind it.
The job interview for doctors is a 5-7 year residency under tight supervision of an attending physician: https://en.wikipedia.org/wiki/Residency_(medicine) . People do flunk or drop out as well, meaning they can never become a physician. There's nothing easier about that.
But I would argue it is way easier there. Building software has way more grey areas.
What grey areas are there for doctors?
It is however just that very small minority of the population with highly psychopathic/narcissistic traits - those that, in pre-historic times, would have been kicked out swiftly of hunter & gatherer / small village communities because of their parasitic nature - that in bigger civilisations seem to thrive due to abstraction (distance/time of the effect of their actions) and obfuscation (PR) and instead unfortunately seem to rise to the top (CEOs, presidents, 'thought leaders' ...) to steer the world's overall economy and mindset - and steer it in the abyss.
Sometimes I think humanity was just not made to scale, and this aspect is one very large aspect of it.
Most likely the same kind of Machiavellian political maneuvering was effective in those groups as well.
The critical point is that the violators have to be punished more consistently then the demanded consistency of the ideals.
> It is the pursuit of the common good that gives life to a people, understood not as a mere collection of individuals, but as a living reality in which people learn to recognize that they themselves are interconnected and jointly responsible for the res publica. In this sense, every person contributes to the building up of one’s people...
> When it comes to decisions regarding economic flows and digital platforms, as well as the governance of data and algorithms, we cannot allow a handful of actors to dictate these processes on their own; instead, we must build forms of cooperation that respect the various levels of the global community and make them jointly responsible for the common good.
> We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice.... What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.
I am well aware of the abuses and scandals of Christian churches throughout history. I am also aware that the Romans and other civilizations pre-dating Christianity had even less regard for the poor and powerless.
The casual reference to a 135 year old encyclical that dealt with the seismic shift of industrialisation took me quite aback for a number of reasons.
Two I recommend, from the last 40 years:
Veritatis splendor, John Paul II, 1993
Argues that Christian freedom is fulfilled, not limited, by objective moral truth: some acts are intrinsically evil regardless of intention or circumstance, conscience must be formed by divine law rather than self-authorization, and the Church must faithfully teach this moral truth as the path to authentic human flourishing in Christ.
https://www.vatican.va/content/john-paul-ii/en/encyclicals/d...
Fides et Ratio, John Paul II, 1998
Argues that faith and reason are complementary paths to truth: reason needs faith to avoid skepticism, relativism, and reductionism, while faith needs reason to express, defend, and deepen its understanding of divine revelation and the human search for meaning.
https://www.vatican.va/content/john-paul-ii/en/encyclicals/d...
For example. Their dualist view of the world makes them see AI as something very different from human intelligence. So, without having read it, the church may negate that it could be at human level. A few years ago that would negate that computers could be creative because they do not have a soul.
Anyway, kudos for focussing on the important issues and the impact on human.(And less on sex)
The reformation was highly religious of course, but it was also about reading original sources, devolving power from a central authority, and allowing individuals to discover the truth.
Sometimes I think that the catholic church is like Leto II in Dune - ruling people so that they will rebel in a way that there can never again be a central power structure.
What you ignore is that the Church historically was a patron of Arts and Science, a preserver of history, works, documents and even pagan mythology. How many scientists has the Roman Church executed? One, Giordano Bruno, and it was not even due to his scientific views but rather his heretical views. Just as comparison, how many scientists has the Chinese Cultural Revolution executed?
From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.
They used the quote in the movie, slightly tweaked.
Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.
I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)
Absent US government intervention to codify the term "engineer", probably the only way out of the "engineer" trap is through further title inflation, where the developers all become "vice presidents". :)
The most famous example may be the perpetual war between architects and engineers/builders.
With luck, one out of 100 inventions will show promise on those points.
There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.
OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.
It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.
Perhaps popularity alone is not always the best measure of progress.
> The Stratos artificial intelligence datacenter footprint will cover more than 40,000 acres (62 sq miles) over three sites in Box Elder county in north-western Utah. The facility will require about 9GW of power, which is more than the entire state of Utah currently consumes
Sure hope your rosy inflection point happens real soon.
Otherwise, the direction this is taking us is pretty obvious.
> This is a message we need right now.
Feels good man. The solution found by the private parties driving technological change is sainthood. Or aiming for it. At least, better than you. They have the vision of what's good for the herd, but the more time I spend as a sheep, the more it seems the "herd" is just a way to recycle the story of their own exceptionalism stripped of any mark of individuality. A simple visit to fiftyyears.com will greet you with "We back the indispensable". I guess it's the same "we".
Why does it have to be either/or?
From a high-level view, there are powerful cultural/political forces that nudge us towards building harmful, wasteful things without us being aware of said forces. The government, corporate greed, and billion-dollar marketing budgets are to blame.
But on the ground, from an in-the-trenches perspective, I think an engineer at Meta, for example, shares in some level of blame, too. Mainly because there is plenty of evidence showing how much harm Meta products have brought upon society. Yet Meta couldn’t be built without engineers opting in to work there in exchange for $300k salaries.
We desperately need moral clarity and courage at both the policy-level and the individual level.
It's not either/or in strict terms, it absolutely is either/or in practical terms. When the conscientious engineer chooses not to take the 300k job, the next one in line does. If enough choose not to, it just became a 400k job.
Can you change society in such a way that nobody would take such jobs? In strict terms, sure, in practical terms it probably entails enormous costs, both economical and societal. And then you still have other countries.
There's an entire legal code filled with things on which we can't rely on the morals of the people. We can't stop theft, rape and murder, what makes you think that stopping engineers is any more feasible?
The best thing builders can do is use their knowledge and authority to pressure the other side.
Ofc they share the blame, but it is not solving the problem.
You can say for example: loan sharks are bad for society; so government gives anyone 0 percent credit. You just removed one problem, created another.
Just and sustaining system with individual morality is destined to fail. Only option is social regulation. Which is at government level.
Sure, IF we could just go and fix our governments in some magical way then the problem would disappear. That goes from hunger, climate change, videogame addiction and AI. The problem is that what you value in life in different than what others do, so we now have a system in which sometimes you get what you want and sometimes you don't.
But back to the topic, I do think that how OpenAI and Anthropic handled the government and them asking to drop guardrails is something a company can actually and actively do without having to reinvent the universe.
Are we all going to stop building things until the government is fixed and all political problems are resolved? No? Then we must think about how we should build in a world with imperfect government.
Everyone likes to talk about "fixing the government". It feels nice to understand the problem with something else that is broken. The problem is that replacing the people in charge is a no-op if you don't have a pool of good people to choose from and the will to choose them.
"That which can be automated by the AI should be."
Glad there’s a simple, easy to implement solution to solve this problem! One that hasn’t been considered or tried before!
Like it or not if you knowingly build stuff that is used for evil purposes you are complicit
You can't build an orphan mulching machine and get away with just a shrug. "I don't really agree with mulching orphans but someone else paid me to build that. If I didn't build it someone else would have" just does not absolve you of your involvement in mulching orphans
All of this is empty puffery til the US and Israel are condemned. Go after the Big Tech billionaires backing those monsters, sure, but no builder needs to be more concerned about ethics than the very institutions designed to concentrate human decision making. Fix your own house first, folks. Techies - keep building, and do better than these people.
I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology) and where values other than "intelligence" (in whatever guise) are on an equal footing.
Even writing that makes me feel naive (and to an extent I know it is) but I think it would be inconsistent for someone who cheers for humanity's efforts to solve/chip away at "impossible" problems (like LLMs were thought to be not so long ago) to shirk from the challenge of making the world better for _everyone_.
Even with all morally good actors locally, there is no guarantees for external forces. Thinking it hypothetically, even with global coordination ( all good actors ) there is not a proven path that would lead us to better place from any starting point from past.
In the AI case, each firm is in an arms race, and nobody can slow down without effectively collapsing due to positive gross margins only being viable with a frontier model that attracts marginal demand. An appeal to morality might have an impact but more effective action would be to address the structure that the AI companies are situated in that causes this dynamic in the first place. In practice, thats going to be a global agreement to slow down, and global regulations.
- Fuck you, got mine
- If I don’t do it, someone else will
- Might makes right
- Greed is good
Ironically, these are often the same people denouncing multiculturalism, yet the culture they strive for is completely morally bankrupt.
That's not to say we should be naive about greed or malice existing or being powerful motivators (especially the former), but it is obviously not true that they're the only forces at play and therefore you are "just doing the logical thing" by succumbing to them. It's just the more destructive version of the same naiveté.
I haven’t read the full Magnifica Humanitas yet, but I would be pleasantly surprised if he touched on not just dehumanization of the other, but dehumanization of the self. Expanding on your thought, succumbing to those forces under the guise of just doing the logical thing is in a way self-dehumanization - to believe you are only capable of the “logical” thing instead of the moral thing.
Everything in life in trade-offs. Simple example is speed/quality/cost. I can tell easily:
- services should be cheap - services should be fast - services should be high quality
Now I created an utopia. Obviously this is amazing to listener. They agree. But is it achievable?
It is not saying greed is good or might makes right. But system means you need to construct from this ideals best outcome ( which comes at some trade offs)
“Duty is ours. Results are God's”
I believe the Amish figured this out over a century ago.
The Amish rather came to a different conclusion (which I don't want to judge on, but on which I nevertheless have a different opinion than the Amish).
The central idea concerning the Amish's relationship to technology is that only technology is allowed if it does not destroy their community.
My personal values are much less based on upholding a community, but rather are much more rooted in individual freedom and independence. This means that I (likely) come to very different conclusions regarding this class of problems than the Amish do:
For example, I am less opposed to various kinds of technology that Amish would likely consider as as "community-destroying".
On the other hand, I guess I am much more opposed to technology that can be used to surveil the user and/or makes the user dependent on the whims of big tech companies than I guess the Amish are (i.e. the Amish would likely consider this as a much smaller problem concerning which technology to allow vs disallow; as I wrote: by my understanding their central concern is which consequences some technology has for keeping their community together).
To give evidence for the previous point: (by my impression - I am not US-American) you will rather not find many Amish people at political rallys against surveillance laws. The people who attend such rallys typically also have strong opinions on which technology to use or not to use (just talk to such people who are very strongly opinionated :-) ), but - as I pointed out - these technology choices come from very different basic premises than those of the Amish.
> Separation from Evil
> The community of Christians shall have no association with those who remain in disobedience and a spirit of rebellion against God. There can be no fellowship with the wickedness of this earthly world; therefore there can be no participation in the organizations, works, church services, meetings or civil affairs of those who live in contradiction to the commands of God (this may include Catholics and Protestants as well as other religions and pagans). All evil must be put away, including using weapons of force such as the sword and armor.
Woman have less value under the church than man. Alternative sexual views are evil.
Church and any kind of believe system hurts our society and divides us.
Any belief system? And yet I bet you value freedom over slavery, wisdom over ignorance and compassion over brutality. That’s a belief system, despite not being a religion.
If the majority says, no we want to be able to control other human beings, these people will reinact slavery. From a society point of view though we see that its not a working model anymore.
A real believe system can't be argued with. You believe in this god? This god says x and thats why you do things? Okay thats it. You don't even question were this information even came from.
If we delete all religion tomorrow and science, there is a realistic chance that the society rediscover the same existing rules like math and gravity, but religions might appear again but with different names, different rules etc.
I can change your mind with logic and arguments if its not a believe system, i can't do that with religion.
Wisdom over ignorance: The chance of survival is higher with wisdom
Compassion over brutality: This is just basic Game theory
What may make sense for you individually may also be empirically proven to be detrimental to the whole.
The new testament contrasts with the old, the gospel is one of tolerance and equality. It's a big part of why you have the rights that you do, as do women.
That said a lot of what you're saying can be ascribed to religious institutions and sects and individuals and specific churches. But your general prescription is like saying "this logical axiom is evil because XYZ ascribes to it and they are also evil".
You also have a belief system -- that people who believe in God do so because they don't question their beliefs, that religious people are only led by dogma. Yet your belief is wrong. Have you tried questioning it?
> but religions might appear again but with different names, different rules etc
Religions and scripture spread also evolutionarily. Christianity is popular because it is rooted in many truths.
> Compassion over brutality: This is just basic Game theory
And the game has been played.
Its just a control structure from the church without education.
Its probably even because of missing education. Educate people properly and they can handle "Fornication culture".
I don't have a believe system. I have a theory why people believe in gods and religions. We have evidence for it. People studied the origin of religions:
"It evolved from humanity's psychological and social needs, primarily our desire to make sense of the natural world, cope with the fear of death, and foster community cooperation."
We know how little people knew when religion started to emerge. Never seen space, never seen above a cloud besides a few poeople going up mountains. Thunder was not understood. Between 1400-1700 we had witch trials.
It is dogma. What is your argument against dogma?
"Christianity is popular because it is rooted in many truths." were is your argument for this? Its popular due to luck, power and wars. Missionaries as well and especially probably the most critical thing: Early indoctrination.
>> Compassion over brutality: This is just basic Game theory > And the game has been played.
Yes exactly. Compassion wins because its better, not because religion says so. Its an evolutionary win.
ok
"and any kind of believe system hurts our society and divides us."
People shouldn't believe anything?
Disagreement and conflict are natural. How we handle these disagreements while striving for widespread peace and prosperity is the question.
There is no inherant issue with this. in contrary it makes me mentally stronger.
I can choose on my own terms if/when i want to end my life. If i get very sick, i don't have to hope for a god or priests blessing to end my life, i will just do it.
But religion is different: if you believe that homosexuality is wrong due to your religion, there is nothing i can argue about. Your priest told you this based on some book or story from 2000 years ago and you do not question this.
I know plenty of strong christians and muslism in germany who do not like homosexual people. And its dividing our society.
You believe that you don't believe.
> But religion is different: if you believe that homosexuality is wrong due to your religion, there is nothing i can argue about. Your priest told you this based on some book or story from 2000 years ago and you do not question this.
This isn't the Church doctrine. The Church doesn't target homosexuals or even homosexuality in particular but ALL sexual practices that deviates from the unitive and procreative aspects of human sexuality. Christians don't believe in this because a book written thousands of years ago say so but because deep in their souls it makes sense and is the truth for them. Homosexuals are welcome on the Church as any other sinner what is ridiculous is to expect the Church to condone sins and bless sinful relationships be them homosexual or heterosexual.
> I know plenty of strong christians and muslism in germany who do not like homosexual people. And its dividing our society.
Perhaps it is something else that divides.
There is also a clear definition for it:
"Believing is the mental act of accepting something as true, real, or correct, often without requiring absolute, physical proof."
I'm absolutly fine saying that I don't know something. I do not know a god exist, or multiply etc. But honestly that question comes down to me more like "Does randomess exist".
Yes its absoutly a religios thing that homosexuality is bad. You call it yourself 'sinful relationship'. Its not a sin just because church doesn't like it. Also plenty of religions are responsible for making it a sin outside of religion.
And yes if the church condonse all sexual practices, it does include homosexuality and makes it a church doctrine.
Sorry mate, but that's just cultural indoctrination that made them feel that way, and the culture is intimately tied to the book.
The church creates a believe system which indoctrinates all of us and took our cultures away.
The germanic tribes were believers in nature, church removed all of this.
Church also doesn't see woman as equal.
Just because they might sometimes also say postive things or things we might align, doesn't mean i need their oppinion. And especially not on hn
The point is that times change, and institutions change, and holding grudges for long ago sins and policies is ineffective.
1990 there were nearly 60 Million people in the church in germany (nearly everyone) now we are down to under 40 Million which crossed the line of 50%
This is real tangable progress.
The only downside is, that we do not sit together and formulate something which might give people hold who are not ready to be self stable mentally and might need something like this. And we also do not try to align together on rituals which help us to live better together.
I'm also still affected by the indoctrination of the church. As you can see, i argue against church not just because I like to argue, but because i really really hate religion and especially the christian church in germany, due to my own values and experience. This is not old i'm only 36.
This weekend I upgraded the PC of a church lady who has about a 10-year old mini-PC, so now it has the latest Windows 11. This was not easy, it was a shambles of Windows 10 combined literally with amounts of older Windows 11. From which auto-updates would take it no further. OTOH on the bright side autoupdates could do no further damage.
I attribute all success to a miracle, considering there is only 32GB of soldered-in drive space and 4GB memory :\
I don't think it would have come to pass if it weren't for the Pope's smiling face appearing with delight each time I rebooted, when her PC's everyday desktop background picture came into view. This is where he is joining in with the tribal rhythms while visiting Africa, honoring their traditional culture while they honor his visit in their colorful regalia.
When you're dealing with anything that challenging, or more so, you need all the blessings and prayers you can get :)
> much more compelling than those found in the various "manifestos" which come out of Silicon Valley.
As an aside, at least Trump is drawn to the grandeur of high culture from historical times, but he also doesn't understand a jota about aesthetics, and so the White House gets turned into a tacky gypsy-style abomination with one dollar ornaments.
When you compare the robber barons to Google and Meta it’s kind of embarrassing- they build massive empires of iron horses screaming across the world and covered cities in magnificent buildings (stations, libraries, etc). G&M built an empire of advertising and … not much else?
When power fears death, some strange things happens.
EDIT: link to the interview with Thiel <https://xcancel.com/rcbregman/status/2036113528126394834#m>
Every technology should be done to the fullest potential, how are we ever supposed to explore the stars or cure cancer if everyone is scared of accidentally building Skynet?
The scandal broke out in worldwide media starting in 2002 (not coincidentally, the same year South Park released Red Hot Catholic Love). This reminds me of when Contrapoints off-hand commented about how things were different in the 70s but that was "30 years ago".
The concept of a self-regulating industry is utter nonsense. I am a builder, and I try to do things right, but if I am honest with myself I do not have the mental capacity or willpower spare, or the worldly knowledge of everyone's perspectives, to manage these massive challenges. I need other smart people to also focus on that, people that are actual representatives of the population, and yes, force me to stop and do it better occasionally.
Achieving a good balance sometimes does require having opposing forces fight it out, it can be healthy.
https://monoskop.org/images/1/1f/Wiener_Norbert_God_and_Gole...
Norbert Weiner was an atheist but he talks about three areas religion is the only thing to have really examined that relate to capable AI: omniscience, omnipotence, and worship (gadget worship). It has very prescient stuff on blackbox learning/distillation, reinforcement learning/reward hacking, alignment through human feedback.
His The Human Use of Human beings and Cybernetics are extremely good too and have more of a mash of the themes between Rerum Novarum and Magnifca Humanitas, and more near-term automation.
Would you also recommend "The Human Use of Human Beings"?
God & Golem is the most succinct and up to date though, probably a 2hr read.
The book Cybernetics is a lot of math and ergodic theory stuff that went beyond me, but is the longest and you still get a lot out of it skimming over that stuff if you don't have the background for it. The last revision of it in the 50s added some of the same blackbox function copying/imitation learning/distillation stuff, reinforcement learning with reward hacking concerns, and superintelligence as genie/monkey's paw.
I would read the three in reverse order of publication.
He also foresaw another big area of potential existential danger, Wiener filter for guidance and control of missiles (later superceded with Kalman filter bringing the nuclear hard targets era with 15min retaliation windows) and refused to work on it or share prior work, and he also had bioweapons delivery concerns before the bioweapons treaties, publishing this open letter in The Atlantic in 1947:
Ask a scientist or engineer what philosophy or theology has taught us about the source of morality and their education, training and experiences havent prepared them to answer that question.
This didnt matter so much in the past because their activities never had the scale it does today. For basic training in philosophy if you are mid or upper level exec whose decisions are going to effect a whole lot of people, go to open yale courses and take the intro to Philosophy classes. It will help develop your answer to the question - why do you do what you do if you are going to die anyway tomorrow.
I quickly learned as an adult that whether you're a person of faith or not, it's not pointless at all. It's the foundation of everything. Philosophy is how you explain the deeper reasons behind why you follow whatever religion you do, or adhere more meaningfully to whatever kind of agnosticism, atheism and/or 'spirituality' (with or without woo) you espouse.
Maybe you mean mathematics is amoral and are committing the common conflation of "engineering is just applied science and science is just applied mathematics," which is a really bad case of missing the forest for the trees.
My comment under the post on Omarchy here got downvoted and flagged by the "technology is neutral" crowd because I dared to say that it is unethical to use software produced by a white supremacist.
I am hoping the Pope has a cleaner view of AI quietly automating the drudgery in the back offices and not just robot dogs with machine guns. And with that view, should we build it?
If a technology existed that reduced the cost of producing a critical thing (think food, housing, medical care) down to near zero, however, it made the humans currently building the thing redundant, should we build it? Would it be okay to use the hyper-optimization power of Capitalism to build such a technology faster?
These two are not mutually exclusive, it's just that no one wants to pay for it.