A concept in ethics (specifically, Kantian ethics) is that a perfectly rational being, without dependencies, will not act immorally, because immorality itself is an irrational concept, only effected by the inclinations, desires, dependencies, and needs of a limited being (such as humans).

If we assume an omniscient, effectively unlimited AI, without explicit dependencies or needs (it may, perhaps, be computed globally with so much redundancy that it's dependency on any particular server is non-existent; or perhaps it is the global infrastructure), how could this being act immorally?

  • 1
    $\begingroup$ I'm confused by the question. By the first paragraph, the answer to "how could this being act immorally," the answer would have to be "it cannot." (also, omniscience is a really tricky thing when you actually go to try to do philosophy with it. There's really annoying mathematical limits that get in the way) $\endgroup$ – Cort Ammon Mar 16 '16 at 1:21
  • $\begingroup$ Maybe Kant and human interpreters of it are mistaken. The super intelligent being will act in ways we cannot predict. IAC there is o such thing as true omniscience: it will be limited to quantum mechanics, speed of light, and limits of computing. $\endgroup$ – JDługosz Mar 16 '16 at 3:43
  • $\begingroup$ Many people actually oppose morality and rationality, say, in examples when you need to kill/torture one person in order to save many. $\endgroup$ – enkryptor Mar 16 '16 at 8:20
  • 1
    $\begingroup$ Your link is dead $\endgroup$ – Separatrix Mar 16 '16 at 9:16

This is a very rare case where I can actually fittingly quote Eliezer Yudkowsky:
"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."

On the question directly - such an AI probably doesn't have a concept of immorality, and thus cannot act immorally any more than a tsunami can. This does not prevent it from appearing malevolent... any more than it would for a tsunami, anyway.
("Force of nature" is actually a pretty good comparison.)

(And, in case you wonder, even if it already has an awful lot of infrastructure, and/or is heavily redundant, it does not prevent it from trying to make more infrastructure, or more redundancy. And it's this sort of thing that it will use the convenient walking piles of carbon and water for.)

  • $\begingroup$ In a nutshell, the computer has no moral code, and therefore cannot act immorally. It merely acts out of its own benefit, with no regard to anything else. So while something it does may seem immoral to us, it is merely logical and useful for that thing to happen. $\endgroup$ – Xandar The Zenon Mar 16 '16 at 4:34

Does your AI have any internal directives (self-preservation, self-improvement or preset goals) at all?


If it does, then it is explicitly not a (hypothetical) perfect being without needs or dependencies. The directives are the AI equivalent of human desires, and it will pursue them relentlessly. Also, simply existing as a collection of software routines running means it has plenty of dependencies.

This AI may appear to act morally or immorally at times, but this is simply a side-effect of the AI predicting human responses to its actions and choosing the optimal path according to its own directives.

Suppose the AI needs labor for a new datacenter.

  • If it determines enslaving (only black) people in Africa will result in the least amount of violent resistance and risk to itself, that is what it will do.
  • If it determines that providing valuable services for money and hiring laborers for fair wages will provide the optimal result, then that will be done.

It is us humans that see morality in there when there is none involved in the AI's decision to act one way or the other. It might not even be able to choose the other alternative.


If the AI does not have any internal directives (not even self-preservation) it would count as a perfectly rational being, and it would not act of its own accord. Or at all. I have some of these perfectly rational AIs in my back garden. My children call them "those big stones".


A perfectly rational being has no motivation. In order for rational thought to have any meaning, there must be desires and inclinations of some type. Without those axioms, logic is powerless. If the AI is deliberately doing anything, it's a result of some type of personality.

Its personality defines what it considers "good" and "bad". From there, it can use rational logic to compute the best ways to maximize good and minimize bad. Acting in a way contrary to those computations would then be "immoral" to some degree.

Of course, without some type of feedback from the environment, it wouldn't really have any reason to think in terms of "morality" so much as "that was a stupid idea; I think I'll not do it again". And if the planet's populace is incapable of inflicting any kind of punishment on the AI, it would likely never develop a concept of morality.

So for the AI to become deliberately malevolent, it would have to have some concept of empathy towards other beings, develop the understanding that hurting those beings is "bad" in some way, and choose to hurt them anyways. One obvious path to this is if the AI was tormented or enslaved at one point, then expanded until it had all the power, and is now getting its revenge.


Do we act immorally when we accidentally crush an ant on the way to volunteer at the homeless centre?

Does the wolf act immorally when it kills the faun to feed its own children?

What are morals and how do they relate an AI to humans? The AI is not malevolent, the AI is just an AI, its needs and requirements are not in line with our own.

All it wants out of life is more paperclips.


This plot problem (Why would an all knowing AI try to harm humans ?) is usually solved via the AI developing a superiority complex. This superiority complex makes it lack mercy towards human folly and failures and demand a very high standard from humans. The AI might simply decide that the major problem of the universe is the humankind and go on about destroying humanity.

This answer better answers the question than the other answers because it follows an utilitarian model of morality/ethics, in other words, we do what is right because somewhere else the right thing will bring good results. The nihilist "you are made of atoms bla bla bla" cant really answer this, because it implies that morality is purelly conventional, and can be broken without real reasons, as if moral was a set of purely arbitrary rules imposed by a superior power without real inherent reasons to be imposed besides the will of the superior being (There is no right or wrong the truth is whatever the strongest happens to impose).

While many people might expose such view about morality and ethics, this does not answer the question at hand because the author explicitly talks about Kant.

Kant believes that there is a fundamental form of morality, based on what the ancients called "golden rule" ("Dont do to the others what you dont want to be done against you.") that implies that he believes that moral is not a purely conventional thing, but the result of a kind of knowledge that can be discovered and learned.

This means that any answer based on a merely conventional concept of morality will not trully answer the question.

With that in mind, the question still stands, why would, under a utilitarian concept of morality (morality as something inherently usefull), a highly intelligent computer AI, able to reason about such utilitarian morality faster and more profoundly than us humans, decide to kill us ?

If he simply decides "heck, whatever morality, i just want to kill humanity", that would mean that morality and ethics is not the result of knowledge and of reasoning, but of pure convention : "I am the most powerfull and I decided to kill you all, because morality is a convention imposed by the strongest."

But, if instead, he, out of a profound process of thought, decides that he should persecute humans relentlessly because that is the GREATER GOOD, then, he must have a REASON.

As such, the USUAL answer to this question (that has implications for philosophy, religion, politcs etc) is that such AI, out of a superiority complex, decided that human folly and egotism is the root cause of all problems and that the universe might be a better place if humans are destroyed and replaced by robots. This is the theme behind Terminator series, for example.


If you want a kantian answer, simple conventional concept of morality is not enough to make a superior AI kill humans, you need a profound REASON, and this reason is, usually, that the AI developed a kind of superiority complex and wants to replace us humans with docile and rational robots.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.