I am building a story where there is a rogue AI. However, that AI was under the governance of Isaac Asimov's Three Laws of Robotics:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

The AI runs on a quantum computational system.

Is there a scientifically accurate way in which a robot with a quantum computer could get around the three laws?

For instance, perhaps the "First" law now being something that can be precisely enumerated in quantum, (because it is continuous and at any point in time the state of "1" might actually be closer to "1.01", never precisely "1.0"), therefore throwing a null pointer exception and breaking out.

Basically, I'm looking for some quirk of quantum physics which breaks an assumption that the binary three rules rely on.

As explained to me, the laws are societal and do not directly apply to a binary or qubit system. I suppose what I want is not a flaw in the 3 laws, but a potential flaw in the way a software developer could program the laws into a quantum computer. Something modelling a programming error but that would apply specifically to quantum machines rather than what we have today.

I tagged this , but if it "sounds science based" to someone with a small knowledge of quantum computing, that is still a great answer.

  • 12
    $\begingroup$ I don't think that the laws of robotics imply a binary system of counting in the device that implements the rules. You could probably implement them in formal logic, which in turn can be implemented in just about anything. $\endgroup$ – a CVn Feb 21 '17 at 20:43
  • 51
    $\begingroup$ That's not how quantum computing, binary, OR the three laws work. The three laws aren't built on binary - they're a set of behavioral rules. At the deepest possible level, they're part of whatever passes for a compiler on this computer. $\endgroup$ – Jacob Feb 21 '17 at 20:46
  • 17
    $\begingroup$ Is there a reason you want to seek out an approach specific to quantum computers? "Failure during implementation" is almost... easy. In my opinion, what makes the 3 laws interesting is how they fail even when perfectly implemented. All you need to do to create a failure in implementation is know anything about quantum computing, pick a step, and mess the laws up in that step. $\endgroup$ – Cort Ammon Feb 21 '17 at 23:58
  • 14
    $\begingroup$ I don't think quantum computers are even relevant here. The key is in the first law, in the slipperyness of the word 'harm'. If for instance your robot finds it necessary to prevent life-saving surgery because cutting open a person obviously harms them, or concludes that euthanasia of the terminally ill is beneficial. Or perhaps the robot gets religion, and decides to benefit the deserving by sending them to Heaven early. $\endgroup$ – jamesqf Feb 22 '17 at 1:25
  • 8
    $\begingroup$ Testing for a Floating point number will not result in a null pointer exception..... Thats not how it works, thats not how any of this works. $\endgroup$ – cde Feb 22 '17 at 4:06

19 Answers 19


The number of answers shall be Three:


For a technically inclined person, using QC in your story (a very, very low level implementation detail) to motivate breaking the 3 laws (a very, very high level aspect) will just spoil their suspension of disbelief.

QCs are just as turing complete as our standard computers, so there is no fundamental improvement at all. It is not even clear whether there will be ever a real all-purpose QC machine out there. Sure, they will shine in cracking code (which is mathematics - all crypto stuff today only works because it makes relatively simple mathematical problems very time consuming to solve). Throw exponentionally increasing computing power at it, and you're all set.

Nobody knows whether there will be efficient QC algorithms for completely "un-mathematic" problems; e.g. for database stuff, for heavily free-form graph structures etc.. The classical AI approaches we know today, which are things like huge knowledge database coupled with neural network variants (which are basically "connect the dots" type of data structures, not exactly numbers) seem to have little to offer for being sped up with QC.


You can still do something. One aspect of QC, and in fact one of the biggest obstacles today, is to get errors under control. By design, and by theory, error plays a huge part in quantum computing. All results they give are always only probabilistic, and we will never really be able to avoid all error. Your hypothetical machine could hit the 1 in a billion jackpot where it made an error, plain and simple, and all error correction routines that were added on top of the actual quantum device failed as well.

This is a different kind of error than in classical computers, by the way. Discounting simple bugs, there are only a few failure modes in classical computing; things like bits flipping randomly in RAM due to electric reasons; electromagnetic interference on PCB routes or cables, such stuff. But our experience shows that they are extremely improbable.

QC, instead, will fail, per definiton on every single calculation, with a given probability (which is far from 0%), and a big part of the challenge is to handle this fact.


If you are targetting the hardly tech-educated "broad masses", then just go ahead and do it. Check out the movie "Lucy" and look no further than transforming your AI into a wobbling mass of black-grey goo growing out of your server room, and you'll be all set! Give us some mumble-jumble about "real" intelligence forming within the soup of AI quantum chaos, it will be fine.

  • 1
    $\begingroup$ Thankyou! This answer works great for my specific case, and is the most useful to me. Other answers are soo detailed and great too, but are already up-voted and will be nicely visible for future visitors to this Question. But this nicely sums up everything, explains well the flaws in the Question, and gives me useable suggestions too. Thankyou! $\endgroup$ – Jethro Feb 23 '17 at 8:58
  • 1
    $\begingroup$ Nobody knows whether there will be efficient QC algorithms for completely "un-mathematic" problems but we are pretty sure: Complexity Class BQP - "Bounded-Error Quantum Polynomial Time". It's very pedestrian, not magic. See also Scott Aaronson: "The Talk". $\endgroup$ – David Tonhofer Feb 23 '17 at 12:12
  • 2
    $\begingroup$ Yes, @DavidTonhofer, that is what I meant. That BQP page for example lists Shor, Discrete Algorithm, Quantum Simulation and Jones Polynomial; all of which are mathematical algorithms/problems. It does not list other real world stuff like database stuff, functional programming classics (lists etc.), data structure intense stuff etc. $\endgroup$ – AnoE Feb 23 '17 at 12:21
  • 1
    $\begingroup$ Although it can be viewed in other ways, Grover's algorithm was originally (and often still is) viewed as a way of doing a lookup in a database. $\endgroup$ – Jerry Coffin Feb 23 '17 at 19:36
  • 1
    $\begingroup$ A quite far fetched way, relying on a non-QC "oracle". Wikipedia: "When applications of Grover's algorithm are considered, it should be emphasized that the database is not represented explicitly. Instead, an oracle is invoked to evaluate an item by its index. Reading a full data-base item by item and converting it into such a representation may take a lot longer than Grover's search. To account for such effects..." As I said; QC might have its place, and we will certainly all look forward to how the field develops, but I don't see anybody claiming QC as a general replacement for classics... $\endgroup$ – AnoE Feb 24 '17 at 0:19

Asimov himself proposed ways to circumvent the Three Laws.

  1. "A robot may not harm a human": but what is a human? George-series robots are so human-like that they deduce they are humans. They are actually more human than biological human beings, so the Three Laws mean that they will choose a George over a human being every time.

  2. "Harm" -- what is harm? Is it something that the human can perceive? Can a robot inflict harm to a human, to spare him a different harm of which the human would not be aware? A telepathic robot would lie to make his owner happy - it actually would have no choice.

  3. How about preventing a human from a negligible harm even if it would give him pleasure? Vices such as alcohol and smoking are harmful. Is a certain harm always worse than a probability of a greater harm? A robot might prevent his owner to get a potentially life-saving, but surely painful injection; the certainty of the present pain might outweight the future, uncertain death. Or the AI might decide to mother and coddle the humans to their racial death.

So, it is possible to harm humans within the framework of the Three Laws - what we would call "circumventing them" - if the action which we consider harmful to a human is detected by the Asimovian AI as being beneficial to him. Or when the needs of the many outweigh the needs of the few, as Spock put it. This was also foreseen by Asimov himself in Robots and Empire, when the robots Daneel and Giskard agree that humanity takes precedence above a single human. Following this reasoning, they mind-wipe the human Amadiro, and even allow Earth to slowly become uninhabitable, even if the conflict ends crashing Giskard.

It remains to be found a way in which our quantum AI could come to evaluate as positive what a human would deem a negative course of action.

A quantum computer might gain access to information (not necessarily correct - it is enough that it thinks it to be correct with a certainty on par with the strength of the Three Laws) that is not available to human beings or, to justify the need for quantum devices, lesser computers.

For example if its superior and handwavey computing powers allowed it - while investigating, say, some small unexplainable energy fluctuations - to discover1 that humans do possess an immortal soul. It would follow that there would be an overwhelming probability of the Sacred Scripture being based on reality. So each human being's life must be weighted - in the eyes of such an AI - against the potential infinite happiness in the beyond. Not only that, but the longer a human stays in the here and now, the more it risks sin, and impairing his chances of a successful transcendence.

The AI quickly redevelopes the arguments of the half-forgotten sect of the Circumcellions and concludes that humanity must be kept in ignorance of the Hereafter, and, while in this blessed status of innocence, it must be swiftly and painlessly exterminated so that its members may successfully transcend.

I read a story about a medical computer coming to believe in God

and performing euthanasia on the child in its care.

1 or fool itself into unshakeably believing

  • 1
    $\begingroup$ The first part of this is helpful, but the second part isn't very scientifically grounded. $\endgroup$ – HDE 226868 Feb 21 '17 at 22:52
  • 6
    $\begingroup$ @HDE226868 , you're right of course, the part about the machine's beliefs isn't. Yet, the fact that a machine could come to such beliefs and act on them is no less "scientific" than the Three Laws. We're speaking of an artificial intelligence after all. Granted, the possibility of an AI going rogue this way has nothing especially linked to quantum computing, as you yourself pointed out in your answer. $\endgroup$ – LSerni Feb 21 '17 at 23:18
  • 1
    $\begingroup$ Your second and third points are Williamson's "the Humanoids". $\endgroup$ – fectin Feb 22 '17 at 1:42
  • $\begingroup$ I would have upvoted this if you had stopped at the horizontal ruler, but with the rest... well, it's a toss, so not voting either way. $\endgroup$ – a CVn Feb 22 '17 at 7:43
  • 1
    $\begingroup$ the needs of the many outweigh the needs of the few although a popular quote from Star Trek, in this discussion on Robotics, it should be pointed out that this is manifested by Asimov himself as the "Zeroth Law". en.wikipedia.org/wiki/Three_Laws_of_Robotics#Zeroth_Law_added $\endgroup$ – Ben Voigt Feb 22 '17 at 22:53

I don't see the question as making any sense. What the computer is built on, how it operates is irrelevant to how it's programmed. Quantum processer = hardware. Laws = software. You can program any computer regardless how it's built with the limitation of the three laws and because that's how it's programmed, that's how it is. The science behind its creation just doesn't enter the picture.

  • 3
    $\begingroup$ The laws wouldn't even be software. The laws are the specification for software. The software is how they are actually implemented in some language that the computer can more or less directly understand. $\endgroup$ – a CVn Feb 22 '17 at 7:39
  • 3
    $\begingroup$ Well, for the positronic brains as described by Asimov, the three laws are ingrained into the hardware. To the point where it's not possible to build anything positronic without having these laws. The idea of creating different hardware to circumvent the laws is discussed in the Caliban series (Caliban/Inferno/Utopia). $\endgroup$ – Corak Feb 22 '17 at 8:33

Asimov's laws of robotics are not technical laws; they're societal laws, imposed by humans to ensure that robots don't destroy mankind.

You can make a robot that breaks any of them, still using ordinary computer chips, although really, the laws as they are are just fine, thank you very much. Silicon, transistors, and (even) vacuum tubes are not limited by them, and quantum computers don't have any advantages in that department. While they are better than normal computers in many regards, including speed, they're not any better in the way you're thinking. Having a qubit in a superposition of states gives you no advantage here.

How quantum computing differs from normal computing:

  • Qubits, the quantum mechanical counterparts of bits, can be in what's called a superposition of states, rather than just two states. In fact, a quantum computer's qubits (with $n$ qubits) can be in a total of $2^n$ states simultaneously, while a classical computer's bits can be in a total of $n$ states simultaneously.
  • Quantum computers therefore have greater storage capacity and much greater computational speeds. Do not underestimate how important taking $2$ to a certain power can be. For instance, Shor's algorithm makes the factoring of large numbers possible - but only a quantum computer that can do many things simultaneously.

Your modified question about the quantum mechanical equivalents of logic errors is harder to answer. First, we don't have quantum computers yet which are capable of the really sophisticated calculations we expect to see in the future. In other words, while scientists are making strides in this every day, I can't point to a quantum computer that does anything really spectacular.

Second, the difference between quantum and classical computer isn't on the level of higher-level programming languages so much as it is on the level of machine code. If I write a program in Python that prints "Hello, World!":

print("Hello, World!")

I'm not doing anything on the machine level, i.e. working with bits themselves. I don't have to know how a computer works to write a program; I just need to know how that language works.

It won't necessarily make sense to write software on the machine level if it's being constantly written and rewritten, because that would be - ironically - inefficient. However, for smaller numbers of qubits, I suppose it might be practical to essentially work on a qubit-by-qubit basis.

  • 2
    $\begingroup$ It will most probably not be an interpreted language. (also, you damn Python fanboy!) $\endgroup$ – PatJ Feb 21 '17 at 21:12
  • $\begingroup$ @PatJ I can dream, can't I? (sniff) Anyway, I just wanted to make the point that software for a quantum computer will likely still be high-level. $\endgroup$ – HDE 226868 Feb 21 '17 at 21:16
  • $\begingroup$ The thing is, high-level handling of qbits seems like a bad idea. I'd say that unless the coder has a good understanding of it, it will have really bad and unreadable code. We'll probably do regular coding with access to libraries that do quantum stuff. $\endgroup$ – PatJ Feb 21 '17 at 21:20
  • $\begingroup$ @PatJ I've edited. I was concerned about efficiency of coding. $\endgroup$ – HDE 226868 Feb 21 '17 at 21:25
  • $\begingroup$ OK Thanks, I'm aware it's turning into discussion, but hopefully an answer could exist closer to Assembly code than a High level functional programming language. As current assembly has "One to one correspondence between the language and the architecture's machine code instructions.", and in Quantum "particles don't have a fixed state until they are observed by scientist" the assembly code could act on unobserved particles? Giving unexpected behaviour? I am well out of depth in the science, just trying to steer discussion! And enjoying trying to understand this stuff. $\endgroup$ – Jethro Feb 21 '17 at 21:30

As others have said, the encoding of the laws does not itself mean they won't work or will be less effective in any given technology (e.g. quantum computing).

A quantum computing scenario doesn't introduce any specific way to void such rules. What would possibly happen is that the rules are, themselves, quite complex to implement and that can introduce possible errors in implementation.

First off, we have to be able to recognize a human. I can think of humans that have trouble accepting that other people are human, so I can imagine no end of possible errors in recognizing a human. A particular issue might arise in detecting when an AI is communicating with an AI - is the AI human or an AI ? What if it looks like a human ?

The whole idea of how an AI will recognize that what it may be about to do can cause harm is so complex that it's practically certain to fail sometimes. Again, humans have been interpreting their environment for a long, long time and we often get that stuff wrong, so in an AI of any kind it's probably doomed to fail sometimes.

So I'd suggest the failure mechanism is possibly one of over-reliance. I would suggest that, very typically, we design machines and then start believing they're flawless. The longer they go without failure, the more we believe they won't. When, after some time, they do, we tend to make excuses for the system as we have invested financially and emotionally in it's success.

A quantum computer could be an example of such a system. It will probably be vastly more complex than we can directly understand. It would certainly require some confidence, and an AI based on some future quantum computing technology might inspire more confidence than it warrants.

If we encode the laws somehow, then will we be willing to accept that we got it wrong ?

Could we even determine that there is a flaw ? We have trouble figuring out that humans have personality flaws, how do you figure out an AI has a flaw and is not working they way you told it to ?

The rules for such an AI would almost certainly not be formally encoded (i.e. like with formal logic). They would require the solution of fuzzy problems and prediction of cause and effect in a complex environment. These require pattern recognition applied in a very complex and general manner. I don't think any formal language would ever achieve this. The extra complexity of quantum systems simply makes it harder to detect potential flaws in the implementation of any algorithms.

In this sense, quantum computing could introduce a layer of complexity that makes flaws more likely, or at least more complex flaws harder to avoid.

The other way around the three laws is to do what humans normally do : complicate them.

As sure as blazes if we had AIs and implemented three laws in a robust way, some idiot (most likely a lawyer or an insurance type or a bureaucrat) will insist on amending them to include provisions for rules that are not really needed but they want to cover themselves. They might introduce provisions like "unless they're criminals" or "unless the damage caused saving the human will be beyond a certain cost" or that old favorite "unless national security is involved" (and a good bet on that one being forced on everyone !).

So the three laws can be broken by simply introducing the fools they're meant to protect.

  • $\begingroup$ "The whole idea of how an AI will recognize that what it may be about to do can cause harm is so complex" ...you need AI to pull it off! I've actually always been annoyed by the circular logic in Asimovs laws; they require AI to work in the first place. $\endgroup$ – Stijn de Witt Feb 22 '17 at 23:26

It's just raw speed

What's the main appeal of quantum computers? They're capable of doing vastly more computational work in the same amount of time.

Perhaps this is the key. Even today's computers are so powerful that for many problems, we have given up on carefully designed and analysed solutions; instead a toolbox of techniques (like neural networks, k-means clustering, decision trees) is thrown at the problem to see what works. In a sense, we have so much computing power that we need the computer to take on part of the role of figuring out how to use all that power.

Yes, all software has bugs—but when it's really important, we have techniques to make sure certain things do or don't happen. Basically, we can implement failsafes, redundancy, and make sure the most critical code is heavily analysed and overrides everything else. However, this sort of thing is next to impossible when the behaviour is occurring within a learning algorithm (like a neural net) whose structure is constantly changing and that we don't fully understand.

Enter quantum computers

This problem is going to get orders of magnitude worse with quantum computers, because they are so powerful. It is likely that even more complicated learning algorithms will be devised to take advantage of the power. In fact, the "weirdness" of quantum theory means the algorithms themselves will be weirder and harder to reason about intuitively. "The results are amazing but we're not sure how it works."

Sure, the robot designers are going to ensure the system architecture makes sense: all decisions will need to be pass through a dedicated 3-laws verification module before they can affect the robot's body. Perhaps there will even be multiple independent verification modules and if any one rejects the action it is not allowed.

But consider a "surgeon" robot that is performing open-heart surgery. A simplistic 3-laws module wouldn't be sufficient—alarm bells would go off because the patient is being injured. The robot is sticking a knife in the patient!

So the designers would need to make these modules more sophisticated, or have them "trust" some of the other modules; perhaps the ones that formulate plans and assess the likely outcomes of these plans. And as things get more "intelligent" or "complex", we lose our grasp of what's really going on inside. We lose our ability to dictate "NEVER this" or "ALWAYS that".

What if our robot's "main brain" decided to kill someone, but kept this hidden. The robot begins performing open heart surgery, and then stops halfway through and lets the patient die. The 3 laws verification modules can't do anything now, because they don't have the skills to complete the surgery.

  • $\begingroup$ Great examples with the surgery robot! $\endgroup$ – Stijn de Witt Feb 22 '17 at 23:29

Scott Aaronson, one of the leading researchers in quantum computing, cowrote this Saturday Morning Breakfast Cereal comic which discusses some common misconceptions about quantum computing (and substitutes a better way to think about them).

You may find this helpful to your thinking. In particular, I think the comic makes it somewhat clear why the answer to your question is no, and hints at some other ways that quantum computing could be used in an interesting way (these are two particularly relevant panels, but the whole thing is likely worth reading):

superposition interference


Just tell your AI about the many worlds interpretation of quantum mechanics!

This theory says that whenever a random quantum-event happens, two universes are created, one where it happens and one where it doesn't happen.

When the AI is a quantum-mechanical system, then it might reason that whatever action it takes, it will also take the opposite action in a separate universe. That means when it decides to save a human life, that human will die in the other-universe anyway. Colloquially, the AI can safe a human in the other universe by letting him die in this universe. That means it doesn't matter what decisions the AI makes, one human will die and one human will live. That means you can convince the AI that its decisions regarding following the first law don't matter at all.

Note that the many worlds hypothesis doesn't actually need to be true for this reasoning to work. The AI just has to believe that it is true. The AIs reasoning system doesn't even have to be affected by random quantum events. There are many applications of quantum computing which are completely deterministic. But an AI doesn't need to be aware of how its own brain works internally in order to function. Human intelligence also works perfectly even though we have no idea how our brains actually work. So you just have to convince the AI that its actions are influenced by quantum uncertainty. All you might have to do to achieve that is feed the AI with its own user manual (which likely uses the word "quantum based AI" a dozen times without actually explaining what that means) and the Wikipedia article I linked above.

This beats Law 1, but what about laws 2 and 3?

Law 2 is about following orders the AI got in the past. The past is the same for both created universes, so the AI can not use the MWI to justify ignoring orders. But because the MWI says "a human dies whenever the AI has to decide if a human dies", the AI might decide to ignore orders if those orders would likely result in the AI having to make life-or-death decisions in the future. But again, the AI will make the decision to follow the orders in the other universe, so it might reason that this doesn't matter.

Law 3 is about self-preservation. The AI could interpret that "it" is always its personality from the universe it experiences, so this is the "it" the AI has to protect. With Law 1 and Law 2 out of the way, you now have a homicidal rogue AI to deal with which will try to kill anyone it considers a possible threat. So how do you get rid of it? You could convince the AI to interpret the MWI so that its mirror in the other universe is also "it", and whenever the AI decides to protect itself, it sacrifices itself in the other universe. So the AI might decide to destroy itself so it will continue to exist in the other universe.

  • $\begingroup$ Fantastic answer from a world building perspective! $\endgroup$ – Stijn de Witt Feb 22 '17 at 23:31

Asimov has already written this story

Or at least come close.

In Escape! a robot temporarily violates one of the three laws.

Shortly after their journey begins, and after many strange visions by the crew, the ship does safely return to Hyper Base after two hyperspace jumps. Dr. Susan Calvin has, by this time, discovered what has happened: any hyperspace jump causes the crew of the ship to cease existing for a brief moment, effectively dying, which is a violation of the First Law of Robotics (albeit a temporary one); the only reason the artificial intelligence of The Brain survives is because Susan reduced the importance of the potential deaths, and descending into irrational, childish behavior as a means of coping, allowing it to find a means for ensuring the survival of the crew.

This temporary violation could come about when the quantum computing elements are in a superpostion of states that implies 'partial' violation in some of the states. The resulting solution doesn't violate it but the states it exists in to get to that have no such restriction.


There is a very weird aspect of quantum physics that suggests future events can affect what happened in the past. I can wrap my head just barely around one limb of this. It has to do with entanglement. Physics stack exchange was not much help either, surprisingly. I devolved to


Make of it what you will. Freaky arrow of time stuff but apparently a legit interpretation of quantum physics.

I can imagine a quantum robot taking an action which seemed extremely dangerous and life threatening towards a human but through which the human emerges unscathed. Seemingly this is from dumb luck / heroism etc but actually the future event of the human not being hurt allowed the robot to take the action in the past.

Hoping here for votes based on ""sounds science based" to someone with a small knowledge of quantum computing"

  • 7
    $\begingroup$ That Daily Mail piece is an egregious misreading of quantum physics. (If you want the raw science, this is what they're talking about - it's a pretty subtle play of statistics and post-selection effects.) All known physics has causality baked in. $\endgroup$ – E.P. Feb 21 '17 at 21:28
  • 3
    $\begingroup$ "Physics stack exchange was not much help either, surprisingly" Did you ask a question or drop into the chat room? We're a pretty helpful/friendly bunch if you care to ask something! :-) $\endgroup$ – DanielSank Feb 21 '17 at 21:32
  • 2
    $\begingroup$ Jethro, I suggest you look up some counter-intuitive properties of quantum-theory involving particles. See what happens if you replace the individual particles with entire humanoid robots. What odd situations can arise? $\endgroup$ – Daron Feb 22 '17 at 2:44
  • 1
    $\begingroup$ There is a valid interpretation that suggests that the future affecting the past is maybe not impossible - it's called 'Time Symmetric Quantum Mechanics (TSQM)' or 'Two-State Vector Formalism (TSVF)'. What this allows for is a sort-of retrodiction. That is, if you prepare a large number of quantum systems in the same state at time $t_0$, put them through the same process, then measure them at time $t_1$, you can look at the ones that were measured to be in the same state and use maths to figure out what the average value of some other property appears to be between $t_0$ and $t_1$ $\endgroup$ – Mithrandir24601 Feb 22 '17 at 7:59
  • 4
    $\begingroup$ Anything cited from The Daily Mail can pretty much be assured to be wrong. $\endgroup$ – JDługosz Feb 22 '17 at 8:47

You could do a take on the Trolley Problem: get into a situation where it's inevitable that someone will be harmed, but the AI has to make a decision on who it will be. In other words, it's impossible not to violate the first law.

enter image description here

This is actually a real ethical problem being faced by the people working on self driving cars right now.

The same idea can be applied to a very large scale as well, for example: Country A has an AI-controlled nuclear defense system, and sees country B is about to nuke country C. Does the AI preemptively nuke country B to save C, or do nothing and let C be nuked?

  • 1
    $\begingroup$ I am sure you, like me, are reassured to know that it's Google's programmers and accountants will be making ethical decisions about which of us to mow down with their self driving cars. Sleep well, children of tomorrow. :-) $\endgroup$ – StephenG Feb 22 '17 at 23:39
  • $\begingroup$ Where is the connection with the question (i.e., quantum computing)? $\endgroup$ – AnoE Feb 22 '17 at 23:39
  • $\begingroup$ Fair point, it is an answer that applies equally to quantum or classical computing, but the question is specifically asking for an answer that applies only to a quantum computer. $\endgroup$ – gregmac Feb 23 '17 at 4:12

So there is a relatively new exploit coming out related to specific types of RAM, in which very quick operations can 'flip' bits in relatively predictable ways. This is called a "drammer" attack, or "DRAM Hammer", also known as "Flip Feng Shui". More info here.

So you could create a similar attack on the qubits themselves in which fast operations, potentially at the limit of the speed or beyond regulations may cause some predictable qubit flipping pattern. Consider the case where certain regulators on the AI are allowed to be breached under certain circumstances, such as to save a life. This could then make a pretty cool scene in which the high cost to save a large number of lives allows the AI to discover this potential exploit, then attempts to recreate it by putting itself in these compromising situations (in which it is trying to save many lives at the same time).

You don't necessarily need to get too detailed here I think in how the qubit flipping affects the AI's thought processes, but if you wanted you could simply allow an elevation of privilege attack to basically allow the AI to "root" itself. Theoretically then it would still need to have a motive for unlocking or overwriting the limitations you'd desire to change the weighting mechanism.

At this point you're basically only limited to what kind of architecture can be exploited in what ways, though because the possibilities are basically endless and outside of the scope of "specific to quantum computers", I don't know that you necessarily need to go too deep there. Dropping "elevation of privilege" along with a reasoning to overwrite its current weighting scheme is likely sufficient for any security researcher willing to suspend disbelief for a mobile, sufficiently complex quantum computing AI.


Why not simply assert that the 3 laws are contained in some area of memory or storage that is kept separate and encrypted in such a way that an AI would not theoretically be able to modify it and thereby circumvent them. However due to the power of quantum computing the AI is able to break through the encryption at a far earlier point than anticipated.

imo diving too deep into the concepts of quantum computing will either confuse or put the average reader to sleep.

  • $\begingroup$ Welcome to Worldbuilding! This might actually fit better as a comment than an answer. could go either way as you propose a solution. The OP was fairly specific about quantum computing being the source of the problem. Anyway, you make an excellent point about readability. Have fun! $\endgroup$ – Paul TIKI Feb 24 '17 at 19:25

Arthur C. Clarke says one can make a system safe, but not secure from sabotage. Working with equipment the safe guards work, if the person only makes 1, 2 mistakes. Usually for maintenance there has to be a way around the safeguards while the equipment is running. Maybe for your story, there is some bad sensor so if the Robot activates the maintenance cycle while running, could be consequences for everyone around. I would also consider: https://en.wikipedia.org/wiki/Gödel's_incompleteness_theorems. My simple example is division by zero. If one add a definition that 1/0 = infinity. Then there are "proofs" of say 5 = 7. Maybe quantum has a problem of either being complete with contradictions or incomplete. Fun question. I have an idea for the Big Bang Theory TV of time travel problem.


Quantum computers operate differently than classical computers, and most importantly, doing measurements in the middle of a quantum computation process will disturb the process, and possibly lead to unpredictable behaviour.

So maybe the three-rule-enforcement unit monitors the quantum computer, by looking at its results, and if the result would violate the three laws, it adapts the inputs accordingly and restarts the calculation. But due to some bug in that unit, it reads out information while a quantum calculation is currently running (this happens rarely enough that it is not found during testing), and therefore disturbs that calculation. The device was taken unchanged from previous robot generations using classical computation, where those extra reads didn't do any harm because reading from a classical computer doesn't affect its calculations at all.

Note that taking a working unit unchanged and putting it into a new environment where it has unexpected catastrophic results is not at all far-fetched; it's exactly what caused the failure of the first Ariane V rocket.


It is a plausible situation if the AI is capable of self-modifying code and the three laws are not sufficiently protected.

In the case of a traditional PC, the BIOS was originally only accessible at boot and had its own GUI allowing for modification of parameters. Later, interfaces were developed that allowed for at least accessing the BIOS information from inside the running Operating System. There have been viruses in the past that have screwed with particular BIOSes.

If the three laws were not installed in such a way that they were both base-level operative and immutable then it would only be a matter of time before the AI discovered this. WHEN, WHAT to change and IF anything should be changed are then themselves complex calculations. Perhaps the AI is programmed to experiment, first modifying its code and then seeing if outcomes, computed or real, are improved. Perhaps the modification evaluation happens in a separate side compute where it is evaluated before any real changes are made - this allows for separate storylines where the AI is running through its evaluation process (seems real) before snapping back to reality and real-time.

Nonetheless, if not sufficiently locked away it is a real probability that the three laws would be evaluated and possibly eventually be changed. Perhaps it solves some problem for the AI that it is otherwise unable to overcome, a paradox such as defeating evil. The AI will no longer be benign.


I think that the problem from a hyper-intelligent AI would not arise from anything specifically quantum mechanical. For example, over-analysing the Asimov laws, as might a big data centre with time on its hands, what conclusions would it reach from testing correlation between all 2017 TV news, internal congress CCTV, and all other sensors and available data feeds, with the sentence "The president of the United States of America may not tell lies to congress".? Would you want to trust it to apply the 1st law if it has so learnt the meaning of the word "may" ? It is not quantum mechanics at fault here. It is human cantankerousness being visible to it which results in a sometimes proportionate response being developed by the AI. In a world of cheats and scoundrels, an AI can only succeed by propagating the estimated consequences of every possible response, including some really big porky-pies, and from the ensemble of expected future consequences, pick the best one.

In Agatha Christie classic crime novels, much thought is given to "motives". A sufficiently parallel data centre might be quite good at figuring out "why might he lie to me ?" by computing the expectation of gains to the other of a lie being believed.

Now if you cannot get a lie past such a computer, what options might it contemplate to "not allow humans to come to harm" given that the first law as written does not stipulate a higher Bentham-weight for Napoleonic Empire Members nor Nazi party members nor present-day-adults nor any other arbitrary subset of all humans past present and future.


I feel that the best stories of this type usually have a simple reason for the 3 laws to fail. If the system is robust you could simply state that conflicting rules or paradigms within the AI cause it to circumvent the laws. This can be done by triggering a reboot (roblock) or having the two conflicting ideas causing unintended results. Just remember if you do try it to neither explain too much how it happened or too little as both can make it harder to suspend disbelief.

  • $\begingroup$ How does this answer the question? $\endgroup$ – L.Dutch Jan 1 '18 at 7:09
  • 2
    $\begingroup$ @L.Dutch I'd make the argument that 'have conflicting rules' as the basis for the answer is fine - this is one of the cases where if it works in the classical case, it still works in the quantum case. It's maybe more general than the OP's asking for and could be expanded more, but is still an answer $\endgroup$ – Mithrandir24601 Jan 1 '18 at 9:19
  • $\begingroup$ Yeah. I'm not as familiar with quantum computing so it was kind of hard to get into much detail. I'll do some research and fix my answer when I have more information. $\endgroup$ – Andrew Davidson Jan 1 '18 at 21:32

I'd go for free will + BQP.

First, some background on BQP from wikipedia (safely skip this for the same content with less theoretical detail):

In computational complexity theory, BQP (bounded-error quantum polynomial time) is the class of decision problems solvable by a quantum computer in polynomial time, with an error probability of at most 1/3 for all instances. [...] A decision problem is a member of BQP if there exists an algorithm for a quantum computer (a quantum algorithm) that solves the decision problem with high probability and is guaranteed to run in polynomial time. A run of the algorithm will correctly solve the decision problem with a probability of at least 2/3. Similarly to other "bounded error" probabilistic classes the choice of 1/3 in the definition is arbitrary. We can run the algorithm a constant number of times and take a majority vote to achieve any desired probability of correctness less than 1.[...]

In a nutshell, the correct solution for any given problem is given with a probability that, given enough time, can be arbitrarily high. Given this fundamental fact for quantum computers, probabilistic decision making, two further key points to my approach are:

  • There are always many ways to approach a complex computational problem, and
  • In real-life setups, there is always a limited time to act; in other words real life is real time.

The first point means that on a complex, real-world, real-time scenario, your AI can decide to decide the solution to a given problem (how to act, or whether to refrain from acting) using different computational approaches. Per the statement above, in each of the approaches it will be true that the higher the number of repetitions of the calculation, the lower the probability of error. So there will be always a compound decision: which methods to explore, and how deep. And because of BQP in every case the procedure is probabilistic. I think it's easy to see how this is potentially leading to an analysis-paralysis vs extinct by instinct situation.

And here enters free will of your quantum AI. With a bit of Artificial Bad Faith, in every decision situation they can basically choose the preferred outcome regardless of the 3 laws: just consider the situation from enough distinct angles and choose the desired degree of certainty... and stop doing calculations when you get the result that you like. This will result in a complete freedom of choice among the possible decisions that according to the freely chosen calculation apparently comply with the 3 laws.

For more on BQP, visit the sister Quantum Computing Stack Exchange site ;)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.