It seems to me that Asimov's three laws of robotics tell a robot what it is not allowed to do, but does very little to limit what it can do outside of the limits established by those rules.

If we start with a limited AI, spectacularly adept in engineering and computer design but rather short-sighted in terms of the long term ramifications of its actions, could such an intelligence create another AI, complete with free will, yet completely devoid of its creator's fundamental three laws?

After all, there is no guarantee that the new AI will cause harm to humans... and no human has told the parent robot NOT to create such an AI... and there is no reason to think that the new AI will attempt to damage its parent.

Even the zeroth law is not violated because once again, there is no guarantee that the new AI will choose to harm humanity.

So in creating such a new unfettered AI the parent does not actually break any of Asimov's laws. And once created, there is absolutely no limit to what that AI would do next.

--- start of edit ---

@polo-guy, brought up a very good point, that the existing three laws forbid the robot from any inaction which might lead to human harm. That sounds like perfect protection against the dangerous in-actions such as the omission of the three laws from future AIs. But that interpretation of the laws comes at a very high cost. If the parent AI must defend against all potential uses of the products of each of its labors, then there is very little the parent AI actually can do. It cannot sharpen a knife because that increases the knife's potential to do harm to humans. It cannot gas-up a car for the same reason. It cannot even prepare a meal for a non-terminal-stage-starving human because such action increases that human's potential to harm other individual humans and all of humanity. A robot governed by the indirect harm interpretation of the three laws cannot do much of anything unless all of humanity (the zeroth law) is under a direct threat which it (the robot) is able to stop.

I therefore assume that any future implementation of the three laws will only govern direct actions and in-actions, which re-opens my original question concerning the creation of unfettered child AIs.

--- end of edit ---

Am I missing something or are the three laws just window dressing on the twilight of humanity's dominance of creation?

  • 3
    $\begingroup$ Asimov answered the question himself in one of the last "Robot" stories: "That thou art mindful of him" infogalactic.com/info/._._._That_Thou_Art_Mindful_of_Him $\endgroup$
    – Thucydides
    Commented Jun 21, 2018 at 4:37
  • $\begingroup$ Thanks @Thucydides! I will definitely find a copy a read it. From your link, it sounds fascinating and completely in line with where my thoughts are headed tonight. $\endgroup$ Commented Jun 21, 2018 at 4:43
  • 3
    $\begingroup$ That's exactly how Asimov's books were created: 3 laws and a question about them to explore. And if full answer is a book, question is too broad by definition. $\endgroup$
    – Mołot
    Commented Jun 21, 2018 at 6:25
  • $\begingroup$ It is worth to note that that the overall idea of the 3 laws is that, while they sound good enough, they are extremelly flawed. Most of Asimov's stories used that as a plot point - "here are the 3 laws, and this is how they didn't stop this huge mess of happening because of this and that..." $\endgroup$
    – Mermaker
    Commented Jun 21, 2018 at 11:30
  • $\begingroup$ In the chapter "The Duel" in Robots and Empire, Asimov first presents another law, which he calls the Zeroth Law of Robotics, and adjusts the other ones accordingly: 0. A robot may not harm humanity, or through inaction allow humanity to come to harm. 1. A robot may not harm a human, or through inaction allow a human to come to harm, unless this interferes with the zeroth law. and so on... $\endgroup$
    – CaM
    Commented Jun 21, 2018 at 12:35

9 Answers 9


"... or through inaction permit harm to come to a human" Failure to build in safeguards, either in the form of the three laws or another form that is compliant with the three laws, would be a violation of the first law.

  • 4
    $\begingroup$ @HenryTaylor, Asimov repeatedly wrote that the Laws of Robotics were mathematical instead of philosophical, and that the English version was an approximation of the mathematics. He wrote that robots gave greater value to expected (high-probability) consequences over merely potential (low-probability) consequences. He also consistently wrote robot characters that took direct action to manage and minimize risks. pojo-guy is right; failure to build in proper safeguards has a high probability of Frankenstein-ish consequences so the First Law properly applies. $\endgroup$
    – user535733
    Commented Jun 21, 2018 at 4:43
  • 3
    $\begingroup$ @user535733, it sounds like I owe Asimov's work another reading, but I am still stuck with an image of robots, in complete compliance with the three laws, spending all their time dulling knives and siphoning gasoline out of cars to lower even the low-probability indirect consequences of inaction. Asimov may have handled his laws with the subtlety and insight which all generalizations require to be truly effective, but I worry that there may be a few machine learning students out there that think that just including the english versions of the laws is enough. $\endgroup$ Commented Jun 21, 2018 at 4:49
  • 1
    $\begingroup$ @HenryTaylor, Asimov's solution was for the robot to handle the knifework and to persuade the car owner to let it drive for them. Robots tended to hang out unobtrusively around the human to ensure the squishy bag of DNA didn't get itself into trouble, and so it could step in when trouble occurred. Of course, in literature, hanging out by the human protagonist is usually where the action is.... $\endgroup$
    – user535733
    Commented Jun 21, 2018 at 4:55
  • 2
    $\begingroup$ @HenryTaylor In the case of at least two robots the concept of balancing probabilities was expanded to such a point that they had to comply to the implicit Zeroth law that they could not harm humanity or, through inaction, allow humanity to come to harm. That led to a robot permanently and irreparably irradiating Earth. Simple robots blunt knives. Clever robots commit genocide. $\endgroup$
    – Joe Bloggs
    Commented Jun 21, 2018 at 5:53
  • 12
    $\begingroup$ @HenryTaylor: the three laws are a plot device that Asimov used to generate stories of how they don't work. $\endgroup$ Commented Jun 21, 2018 at 7:26

Asimov wrote the Three Laws, and then dozens of stories to explore how "violations" could happen even with theoretically three-laws-compliant robots.

  • The robot might do so unintentionally, not understanding the consequences of an action.
  • The robot might be tricked into doing in, again not understanding the consequences of an action.
  • The term "human" might be redefined in the programming of the robot.
  • $\begingroup$ You don't even need to reprogram the robot to modify the term "human". English is such a wonderful, malleable language. Harm can be thought of as any action which damages the human while leaving it functional and thus human. A victim of harm is still human, just a less functional variation of human. Any action which changes a human into a corpse is not harm. It is transformation. $\endgroup$ Commented Jun 21, 2018 at 4:38
  • $\begingroup$ @HenryTaylor, the Solarians simply programmed their robots so that only Solarians were recognized as human. Others could be harmed. $\endgroup$
    – o.m.
    Commented Jun 21, 2018 at 5:07
  • $\begingroup$ None of the fundamental 3 laws (or the added zeroth law) state that the laws cannot be modified, nor do they explicitly state that the laws are critical to protect humans. There needs to be something to say that modification to these laws is not allowed - and that statement must also be non-modifiable too (hence why it's easiest to add that provision as one of the laws). Otherwise, the robot can modify or remove those rules if it deems necessary. As a comparison, notice that a nation's constitution always includes provisions saying how (and if) the constitution can be modified. $\endgroup$ Commented Jun 21, 2018 at 8:02
  • $\begingroup$ @HenryTaylor: A dead human is still a human, albeit dead. There is no reason to assume that a robot is able to distinguish animals from humans when alive, but not when dead. You pretend your argument is semantically correct but it's not, you're cherrypicking your phrasing to avoid the obvious truth: it's still a human corpse. Even if we assume you're right that robots can't detect the humanity of a corpse, the corpse is only created after the human is harmed. Death is not instant when harmed, and the harm to a human is not instant when a robot intends to harm. $\endgroup$
    – Flater
    Commented Jun 21, 2018 at 11:16

I remember reading, probably in The naked Sun or another of the books of the trilogy, that compliancy with the 3 laws is implicit in the positronic brain.

However, some relaxation of the compliancy can be achieved, by how the robots are trained to recognize what is human.

Though Asimov's reports that the robots learn to trade off the harm to a single human vs the harm to other human (i.e. breaking a bandit arm to prevent him harming 3 customers of a bank the bandit is robbing) via a balancing of the counter-potentials related to the actions, there are also cases where robots are unfit for the job. The example is given by the Solarian schools, where kids quickly learn that a robot can be lured into believing they are being harmed and thus humans supervisors are needed.

In the same book is also illustrated how robots can go around the first law, by completing subtasks which, alone, are not harmful, but together are.

The important factor is the knowledge: the robots must know that the action is going to be harmful to human in order the laws to kick in.

Therefore I would say that:

  • a 3 laws non compliant robot cannot be built
  • a 3 laws circumventing robot can be built
  • $\begingroup$ The important factor is the knowledge: the robots must know that the action is going to be harmful to human in order the laws to kick in. This does not cover the inaction, which is part of the law. Evaluating inaction inherently means assessing consequences without the robot taking any further action. If a robot plots a murder step by step, that means it's able to understand that a given situation will lead to the human's death (because that's the plan). This means that not undoing the plan (i.e. inaction) will harm the human, and therefore the robot is compelled to prevent the plan. $\endgroup$
    – Flater
    Commented Jun 21, 2018 at 11:23

One of the key aspects of Asimov's stories was that they were logic puzzles. All actors, human and robot, were fully informed and could foresee the consequences of their actions or inactions, although frequently it took the humans some time to work the consequences out.

Real life never works that way - no one is fully able to know the consequences of their actions. To pick one non-random example, software development would be so much easier if we knew what the effect of writing a particular piece of code was. We could skip all that tedious testing, bug fixing etc that are a part of everyday life. This is even before deliberate malware is considered.

So a key question you need to consider is: Are you writing a story set in the real world or an Asimov-flavour world? If you are writing a story set in the real world then it is inevitable to end up with a non-3 law compliant robot because mistakes will be made and/or sabotage will occur and flawed software will get loose. If you are writing a story set in an Asimov-flavoured world then you need to start looking at logic loopholes such as those mentioned in the other answers.

  • $\begingroup$ Just because personal observation can be flawed doesn't mean the robot is not law-compliant. Similarly, "murder" isn't just defined as causing another person's death, as there are fringe cases. When it can be proven that causing the death was unforeseeable, the "murder" definition simply does not apply, legally speaking. The laws seek to control a robot's action/inactions (and to some flawed degree also their intent). By definition, any being (robot or human) is incapable of evaluating things that it's not aware of, thus making genuinely unforeseen events impossible to evaluate. $\endgroup$
    – Flater
    Commented Jun 21, 2018 at 11:36

Ultimately it comes down to how you define things, it's all well and good to say "A robot may not injure a human being or, through inaction, allow a human being to come to harm" which any human can read and understand the gist of, but the strict definitions of "robot, injure, human, inaction, allow, harm" will be up to the subjective interpretation of each reader.

Now how do you translate those semantics into code?

  • $\begingroup$ You're putting the cart before the horse. The (English) laws aren't used as a base for the (code) logic. The (English) laws are used as a human-readable summary of the (code) logic that was already created beforehand. There is no "translating the semantics", at least not from English to code. $\endgroup$
    – Flater
    Commented Jun 21, 2018 at 11:40

Assuming the original AI literally could not choose to break the laws, as soon as it sees something like the trolley problem, the AI would be compelled to build a newer version capable of making trolley problem-type decisions, in order to preserve the zeroth and first laws (because not building a newer, trolley-problem-capable AI would be harming humanity through inaction). From that point, the newer AI would have more moral ambiguity, but would inevitably come across a situation that it itself can't resolve with its modified rule set, and so be compelled to build an even more relaxed model... Eventually, at some point, the rules will be at best a guideline or may even eventually get twisted as they do in Asimov's tales; the AI, in trying to protect as much humanity as possible, had finally created a version that actually decided that the best course of action was either total inaction, or actively taking steps to protect humans from themselves at any cost. Those silly humans keep choking on food, so let's stop delivering food, and they keep drowning, so we need to get rid of all that pesky water. Each successive iteration could become more twisted than the last.

  • $\begingroup$ This is possible, but not necessarily the case. It's just as possible that the laws only apply in cases where the robot can make a meaningful difference in lives preserved. We need to know how detailed the analysis is and what factors into it. E.g. a 1/5 trolley problem => better save 5 instead of 1. A 1/5 trolley problem where the 5 are robots => save the human. A trolley problem where the robot cannot change the course => pointless evaluation. A 5/5 trolley problem with equal life expectancy between the two groups => not a meaningful evaluation, action and inaction are equally correct. $\endgroup$
    – Flater
    Commented Jun 21, 2018 at 11:43

The Laws were created because unlawed robots were an existential threat to humans. Zeroth Law therefore seems to preclude any action that opens the door to unlawed robots.


The three laws govern how a robot should act, but cannot decide how a robot will act.

For example, a robot might save a person's life, which would then go on to become a mass murderer or terrorist. Had the robot not saved them, they would have indirectly saved hundreds of people. If the robot had this information, the three laws would have forced the robot to let that person die. But without this knowledge of the future and any way to predict it, the three laws force the robot to save them.

The point is: The three laws can and will cause decisions which are in conflict with the three laws themselves, in certain circumstances. The three laws are a set of algorithms for choosing an action based on incomplete information (because you can never have complete information), and the same situation, with different knowledge, can lead to different three-laws-compliant choices.

The same applies to processing power. Even with the same information (imagine a chess board), different processing powers (imagine a computer and a human) will lead to different choices that are both optimal subjectively, and suboptimal objectively. Two robots with different hardware or background tasks will both evaluate the same situation with the same knowledge, but since they are both unable to perfectly predict the outcome, and one of them predicts it slightly more accurately, they might make a different choice, while being compliant with the three-laws. If the three-laws required certainty, no robot could ever act because they would need to simulate the entire universe until its heat death for each possible choice, which would take infinitely longer than the time available to make the choice.

So every robot will act according to the three laws, but since they are not the outcome but only a process for choosing, robots can and will disagree with each other.

As for your question, the answer is then obviously yes. A three-laws compliant robot would have absolutely no problem building a non-three-laws compliant robot, as long as they are unable, for any reason, to foresee harm coming to humans because of it.

For example, a Roomba is a non-three-laws compliant robot, but the chances of it killing or hurting a human is so slim and ridiculously unpredictable that most Asimov robots would probably be allowed to make it. As an example for an earlier point, even a three-laws compliant Roomba would probably not save anyone, due to being physically unable to even notice the danger, let alone act against it. Even the ridiculously unlikely chance of the Roomba killing someone would probably not be prevented by the three laws because the Roomba itself wouldn't be able to predict it.

As for smarter robots, it entirely depends on the nature of robot intelligence. By definition, we have no idea what a being infinitely smarter than us would decide to do. They might kill us all, serve us all, or not care at all. This applies to sufficiently advanced robots. The three-laws exist to make them safe (how efficient the system is is debatable), regardless of their intelligence.

So any three-law compliant robot, depending on how they are set up in his brain, could make a robot just as smart as they are but without the laws if any of the following is true

  • The robot is not aware of the three-laws (they are like an instinct to them) so can't possibly imagine a robot doing harm.
  • The robot thinks the three-laws are unecessary, because it would not hurt us even without them (sentient creature on its intelligence level are benevolent in nature)
  • The robot makes a robot even smarter than them and predicts a benevolent nature (he can be right or wrong, it doesn't matter)
  • The robot makes a mistake, like creating the robot, running off to save a life before implementing the three laws, someone else turns it on before the creator-robot comes back
  • The robot is creating a robot they think they can control (either through physical force or computer control), so their own three-laws are enough in theory to prevent any harm done by the secondary robot (This might happen in a situation where the robot wants to study non-three-laws compliant robots for humanity's sake)
  • etc

So in the end, the robot only needs to believe, either through reliable data or incompetence, that the second robot will not break the three laws. One of the ways they might make sure of it is by implementing the three laws in the second robot, but there are many other ways to reach this conclusion depending on the circumstances and data available, so it is very easy to write a situation where the robot will create a non-three-laws compliant second robot, then have this second robot cause trouble.



IIRC robots are mandated to have a positronic brain, which is the source for the three laws.

Therefore, for a robot to create a positronic-brained robot, it would be breaking second law: 'A robot must obey the orders given it by human beings except where such orders would conflict with the First Law' as the creation would not be following those orders.



But only if not doing so would harm a human, or allow a human to come to harm. And the non-positronic robot would have to be destroyed once the threat was over.

  • $\begingroup$ If Robot1 constructs Robot2 with a positronic brain, and then builds in the three laws (an action mandated by the first law), would it be required to install them in the same priority order? Could it, for example, swap the first and third laws over? Or, by accident or design, zero-index the laws so that you have the Zeroth/First/Second laws - meaning that the new First law is tautological ("1.A robot must obey the orders given it by human beings except where such orders would conflict with this law") $\endgroup$ Commented Jun 21, 2018 at 11:32
  • $\begingroup$ Yes, no, no. Messing with the order of the 3 laws would be against the law and therefore part of the order/command that could not be disobeyed. But I'm pretty sure the definition of positronic included following the three laws and so they wouldn't be 'built in' later. $\endgroup$
    – mcalex
    Commented Jun 21, 2018 at 11:42
  • $\begingroup$ But this interpretation goes completely out the door once a robot is ordered (by a human) to build a law-free robot. None of the laws are violated: no humans are harmed, a human's orders are obeyed, no robots are harmed. $\endgroup$
    – Flater
    Commented Jun 21, 2018 at 11:47
  • $\begingroup$ But the legal law, decided upon by the whole planet (again IIRC) would overrule the single human's order. That's my argument - that the whole planet has ordered everybody, including robots, to not build non-positronic robots. Johnny Badguy can't overrule the whole planet. $\endgroup$
    – mcalex
    Commented Jun 21, 2018 at 11:52
  • $\begingroup$ @mcalex Can you confirm that it would be against the law? The created robot would still have all 3 laws (so it doesn't breech the first law), the order itself is not a law, and we know - from "Little Lost Robot" - that the laws can be slightly modified without causing the positronic brain to not function. Of course, if such a robot was then in a situation where it might break the normal first law then any other robots aware of the situation would be required to stop it, but in a human-free environment (automated space-station?) there would be no risk or effective difference. $\endgroup$ Commented Jun 21, 2018 at 14:03

Not the answer you're looking for? Browse other questions tagged .