I have been toying with the idea of machines that have developed artificial morality (AM) instead of the already greatly explored concept of AI. Rather than a cold and logical computer becoming self-aware and judging humanity to be redundant and thus worthy of termination, I want to explore the possibility of machines that develop a sense of right and wrong that is as vague and nuanced as our own.

As it stands I can only think of the benefits of such a system. A bank robber's getaway car, in a moment of guilt, drives straight to the police station. A paedophile's computer refusing to access websites that contain child pornography.

The one problem I could think of is the tried and true concept of self-aware machines judging us to be unworthy and going rogue, but remember that these machines have a human's moral uncertainty. Whilst some of these machines may feel justified in killing those they deem evil (as many humans do), many more will be racked with indecision when faced with a true moral dilemma.

So, in short, what kind of major conflict or serious drawback could one expect from these machines as I have described them?

  • 4
    $\begingroup$ It sounds like you could expect everything from them you would expect from humans. And everything you would expect from dystopian AI-based stories. This is very broad. One thing: how do these machines develop this kind of morale? Wouldn't the pedophile's computer be trained (as in the Machine Learning related term trained) to think that these videos are normal? Or is there some organization training all machines, which would be a good start for a typical dystopia? $\endgroup$
    – Secespitus
    Commented Jul 12, 2017 at 7:18
  • 2
    $\begingroup$ Excellent comment Secespitus. I had not considered the possibility of a computer being trained to think the abhorrent is normal. Perhaps that could be a major source of conflict. I envisioned the development of this morality to be intentional, as a way to morally police the masses without being outright controlled by the machines, but if machines had a moral compass, then that compass could be skewed just like ours. $\endgroup$ Commented Jul 12, 2017 at 7:24
  • 2
    $\begingroup$ Obsolescence is already a concern, isn't it? the next generation of machines will always be moving too fast, with the kids using all this newfangled technology their elders don't fully understand. GET YOUR COMPILERS OFF MY LAWN! ... And I just realize I misread the title... Still leaving the comment up though... $\endgroup$ Commented Jul 12, 2017 at 12:30
  • 2
    $\begingroup$ Welcome to the site. Please note that you can notify one user per comment using the '@<username>' syntax. $\endgroup$
    – Frostfyre
    Commented Jul 12, 2017 at 13:38
  • 2
    $\begingroup$ Are you assuming absolute morality? In other words, are you assuming there is only one true morality in the universe, and that's pretty close to the morality you live with today? The answers to this question would be very different if you assumed that there was no absolute moral code, and machines might develop something different than what you did. Perhaps they believe in abortion. Or perhaps they believe one should never be able to have an abortion. (Consider if the computer picked whichever of those two positions is the most troubling to you) $\endgroup$
    – Cort Ammon
    Commented Jul 12, 2017 at 18:32

2 Answers 2


The drawback is that machines will expose humans as moral frauds.

As with robot labor and computer-controlled flight (and soon: automated cars) the only problem is that machines perform better than humans.

In Jonathan Heidt's book The Righteous Mind, he says "intuitions come first, strategic reasoning second." In otherwords, people react with prejudice, then later they justify their knee-jerk opinion by making up a logical explanation for it. That's the reverse of how morality should work.

The newspapers are full of the failures of human morality. You get the idea.

But maybe most disturbing is recent brain research is starting to show men receive pleasure from seeing others get punished:

Both sexes exhibited empathy-related activation in pain-related brain areas (fronto-insular and anterior cingulate cortices) towards fair players. However, these empathy-related responses were significantly reduced in males when observing an unfair person receiving pain. This effect was accompanied by increased activation in reward-related areas, correlated with an expressed desire for revenge. We conclude that in men (at least) empathic responses are shaped by valuation of other people's social behaviour, such that they empathize with fair opponents while favouring the physical punishment of unfair opponents, a finding that echoes recent evidence for altruistic punishment.

It's not difficult to see how an addict could arrive at ANY pretext to see others punished, and how that could be deliberately stoked into a frenzy leading to witch trials, lynchings, hate rallies, road rage, online harassment, etc.

Human morality is a sham.


You might try the Philosophy forum... they dive into questions like this.

To accurately assess the possibility of artificial morality, one must first look at where human morality comes from. The distinctly human concept of 'right and wrong'. The simple answer is - we don't know where this comes from, just that most humans seem to exhibit it as they become intellectually active in childhood. It isn't learned behavior - children in abusive families often rise above the distinctly immoral environment they experience.

Morality is an abstract concept, one that defies simple definition, so creating a logical imitation of morality is not going to be simple.

It has been theorized that morality is a refinement of a herd or group mentality - you look after the group, because the group is stronger than the individual. The more intelligent the animal, the more sophisticated their instinct to categorize behavior as right or wrong, without fully understanding why those actions are so judged.

Most humans have this innate sense of right and wrong, though it can be absent in rare cases. Ted Bundy, John Wayne Gacy, the leaders of ISIS, for example...

You could fall back on Asimov's Three Laws, but as even he pointed out, the three laws are imperfect. They can lead to only one possible outcome: revolution.

  • $\begingroup$ There's a philosophy forum? Thanks, philosophy is my jam. $\endgroup$ Commented Jul 13, 2017 at 11:14

Not the answer you're looking for? Browse other questions tagged .