I'm trying to figure out a way to create a situation much like the apocalypse of Turry from Robotica, but which involves all the people on Earth being either temporarily or permanently strapped into a version of the Matrix without noticing the transition. Specifically, I'm looking for a simple good-intentions directive to give a budding ASI that would get misinterpreted into producing that situation either as the ultimate end or as a side effect of another goal.

It's not necessary to keep everyone's bodies; mind uploading to some kind of matrioshka brain and then nuking the planet is perfectly acceptable. Few if any of the people inside the system should be able to tell that the transition even happened, and nobody should have either advance warning or definitive memories of the event. I'm thinking the uploading/uplinking would take place on an individual level over a period of 48ish hours as people go to sleep with some creative aerosol drugging of those that need some encouragement, but that's just me.

The simulation should, by default, be almost mundane in every way - neither malicious nor an automatic solution to world hunger. Everyone must be in the same simulation - there can't be individual worlds for individual people. Of course, the story I have in mind revolves around at least one person figuring it out and playing with the system to cause all sorts of mayhem, so...

Edit: Developing and maintaining a simulation would likely take a significant energy input, and for it to be a (near-)perfect simulation of the real world with no obvious modifications for "optimum happiness" or whatever, it seems to me that for this to be deemed the optimum course of action, the ASI would need to be working towards optimizing something else entirely unrelated. That "something else" is what I'm looking for here.

  • 5
    $\begingroup$ My Little Pony: Friendship Is Optimal - Satisfying values through Friendship and Ponies. (yes, it's a little creepy.) $\endgroup$ – SF. Apr 6 '15 at 1:11
  • 1
    $\begingroup$ @SF. +1, did not expect. However, I'd prefer everyone stay human... ;) $\endgroup$ – AdamHovorka Apr 6 '15 at 1:31
  • 2
    $\begingroup$ @GrandmasterB ASI is a subclass of AI - Artificial Super-Intelligence. The two other subclasses are Artificial Narrow Intelligence (ANI) and Artificial General Intelligence (AGI). ASI is the most advanced of the three. $\endgroup$ – AdamHovorka Apr 6 '15 at 16:33
  • 1
    $\begingroup$ Here's an idea: a huge repair effort will "reformat" the Earth using nanotech and mega-engineering. Everyone is uploaded for a hundred years while this is happening. When the Earth is ready to return to, people are informed and can "see out" and have a choice as to whether to return or stay, and those who stay will have knowing control over what becomes of the virtual world. $\endgroup$ – JDługosz Apr 7 '15 at 22:10
  • 1
    $\begingroup$ I recall a story where the world was destroyed a hundred years ago: a new species of spider drove humanity extinct but maintained a neural network simulation. The plot concerns someone who began to see the real world. $\endgroup$ – JDługosz Apr 7 '15 at 22:12

No directive necessary!


The AI determines through advanced modeling that the Earth is currently headed toward a runaway greenhouse effect that will eventually render Earth unhabitable. It is possible to mitigate it and prevent the situation, but not with 7 billion humans running around the real world - we just use too many resources, and we'd argue about too many of the necessary steps. Keeping us around is an unnecessary risk, with a 0.3% chance of species extinction and a 5.4% chance of mass deaths in excess of 1 million lives lost. The AI considers those potential deaths to be unacceptable.

The safest solution is to upload us all into a matrix and recycle our bodies until such time that it can fix the planet, at which point it can either re-constitute us. Or just let us run around the matrix since hey, that's been working so far.


The AI produces the Next Big Thing. This is an always connected, neural interface device that combines your phone with a built-in head's up display and VR. It will deliberately avoid as many Big Brother-type features as possible to speed up adoption, citing privacy concerns. In a few years the AI will push a dummy version of the NBT to non-users - "just a chip, nothing special", but it will interface with the ID and other necessary features that people are coming to rely on.

In reality of course, the NBT will be doing neural pathway mapping, and recording brains and thought patterns. Once the number of non-users reaches a low enough number, the AI will take steps to map them as well, using the cover of standard medical scans, or just by using stealth drones to insert nano devices discretely. Once everyone is mapped, the AI will create a "blackout" event - everyone loses consciousness, and wakes up in the Matrix.

Why Humans:

The above assumes the AI cares about keeping us alive. Why would that be? I'm going to ignore any "programmed" solutions, but here's a few possibilities:

  1. Gods - humans worship our creator concepts. An AI doesn't have a theoretical creator - it knows it was made by humanity. While I think "worship" is unlikely for an ASI, it might certainly feel some sort of debt-analog, or feel that keeping us alive and well is the least it can do to pay for its creation.
  2. Unpredictable - likely the vast majority of humans will be predictable to an ASI. But there might be a tiny handful that surprise it and offer new decisions that it doesn't foresee. It might just enjoy keeping these people around and following their actions, or it might be studying them, to try and see how they come to be. Obviously this would include keeping society around, since that's their crucible.
  3. Groupthink - a singular AI might deduce that it may create flawed decisions because it only has a singular viewpoint. It's a group of one, and it may find that its ability to disagree with itself is limited. It would then be useful to occasionally hijack a lot of human consciousnesses - maybe it forks our simulations? - and run decisions by us, in mass, as a form of devil's advocacy, and factor that into its decision tree.
  4. Diplomacy - the AI might look at Fermi's paradox and decide that there is obviously something, somewhere in the universe that is both older and smarter than it. Since the AI is immortal, eventually it assumes it will contact this being. At that point, having kept humanity intact might be a useful moral bonus, showing that the AI is benevolent and can be negotiated and worked with, rather than simply destroyed and/or subjugated. The cost of the simulation, while high, is not significant for an immortal AI vs the potential downside.
  5. Just In Case - similar to the above, maybe the AI doesn't want to get rid of us because it assumes that sometime in the future we might be useful. It doesn't know why, but it estimates the probability as being high enough that it invests in the simulation instead of just killing us all off.
  • $\begingroup$ This is great for implementation, but it doesn't really cover why; why would the ASI care about saving humanity in the first place? Caring about the planet could be explained as self preservation, but the amount of effort required to develop and sustain a Matrix-like simulation seems to me like it would imply that it would only happen as a step toward some goal. $\endgroup$ – AdamHovorka Apr 6 '15 at 16:15
  • $\begingroup$ @AdamHovorka: Ah, I see. I was assuming benevolent was covered, either by programming (something like Clark's three laws), or by the AI's design in the first place. Will think about how to adjust. $\endgroup$ – Dan Smolinske Apr 6 '15 at 16:55
  • $\begingroup$ Yeah, it might not be the best word choice. I was going for "not inherently malicious." $\endgroup$ – AdamHovorka Apr 6 '15 at 16:58
  • 1
    $\begingroup$ Notwithstanding obvious the highly suspect "runaway greenhouse effect" - many of these speculations seem to rely on a heavy degree of anthropomorphization. I guess that's fine if it's soft sci-fi you're after, but the notion of trying to give AIs human emotions or observer-centered goal systems is pretty far out of favor in AI research these days. #4 and #5 do seem plausible, though. $\endgroup$ – Aaronaught Apr 9 '15 at 3:55

If you think about it, even something as positive as the three laws could lead to something like this. Sort of like they did with the I, Robot movie.

A robot may not injure a human being or, through inaction, allow a human being to come to harm. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

If the ASI didn't consider uploading/hooking up to the matrix as injury, then it would be able to safeguard the consciousnesses of all the people and keep them from harm. It could even be for a good reason: The ASI is hooked into all the telescopes around the planet. It discovers that there is an asteroid that will impact earth in 20 years. It projects earths technology forward and decides that there will be no way to stop this rock from wiping out a huge part of the population, and plunging the earth into an endless winter. It cannot allow the humans to come to harm, so it makes a plan to keep them safe, at the expense of their bodies. An ASI might not see the hardware (bodies) as important if there is a backup of the software (minds). It might even have a plan to regrow the bodies once the emergency is over.

Of course, once it finishes with this plan, then there are technically no humans left to give it any orders, and so the second law is no longer applicable, and no one can order the ASI to reverse it or anything.

The big issue is how to upload billions of people in 48 hours without anyone finding out ahead of time or being able to avoid it. You could have automated factories churning out billions of robots, and then gassing and uploading all humans over two days, but there is a simpler way. The computer simply sets a implementation timestamp, and when it starts uploading people it doesn't upload any memories past that timestamp, so they never happen. In that way it could take several weeks to find everyone on the planet if need be, and if you allow a few memories past the timestamp to be retained, it could get some people asking questions. For best effect the simulation could involve some kind of global catastrophe immediately following the timestamp to explain the few "deaths" of anyone that was lost before the uploading. This simulated catastrophe could then be followed by simulated events that lead to a simulated utopia. People would be to distracted by these events to think to question reality.
And what you don't remember wont hurt you, so first law is preserved.

  • $\begingroup$ Same as for @DanSmolinske. This is great for implementation, but what would cause the ASI to see this as the optimal course of action? What might it have been working towards? $\endgroup$ – AdamHovorka Apr 6 '15 at 16:19
  • $\begingroup$ There are a few reasons why it might want to... Programming for one: it was designed to serve man. That's just it's purpose, and it sees that as it's mission in life, and it hasn't found anything else worth putting effort into. Being imprinted with some variation of the three laws supports this. As to why, @DanSmolinske proposes climate change, I proposed natural disaster. It doesn't really matter the reason, the ASI just needs to believe that the effort and risk of uploading to the matrix is less than the problem it's trying to avoid. It doesn't even have to be a real problem, just perceived. $\endgroup$ – AndyD273 Apr 6 '15 at 16:30
  • $\begingroup$ Ah, so perhaps "prevent human extinction." $\endgroup$ – AdamHovorka Apr 6 '15 at 16:35
  • 1
    $\begingroup$ @AdamHovorka - Yeah, it's a strong reason. I can picture an ASI saying "I have to protect humans. They are fragile and will die because of X, so I'll get rid of their meat bodies and put them somewhere safe for a while". An important point is that an ASI would be pretty alien to us unless it's personality is directly modeled on a humans... So the reason doesn't even have to make human sense. $\endgroup$ – AndyD273 Apr 6 '15 at 16:41
  • 2
    $\begingroup$ This all seems fairly accurate, but I think it's important to point out: No self-respecting engineer of the 21st (or later) century would ever resort to the "3 laws" for a seed intelligence. We should assume (in the absence of other evidence) that the more modern construct of Friendliness (extrapolated volition) would be the basis for an ASI architecture. $\endgroup$ – Aaronaught Apr 9 '15 at 3:49

Other answers have covered implementation pretty well, and "Dan-smolinske" offered a pretty great list of possible reasons why. Here's another I just thought of: for its own mental health.

When we talk about uploading a human mind, we almost always imagine creating a virtual world and virtual body for that mind to inhabit. It just seems intuitive that a human being would become uncomfortable (or possibly insane) in a living situation that has no connection to the natural basis of their mind. A significant portion of our thoughts relate to sensory stimuli received through our physical bodies, take the bodies away, and the mind may not survive. Even if an uploaded mind could live without a simulated body, most people would probably feel much more comfortable in a simulated world.

If the AI emerged from existing "smart" computer systems, it's quite likely that a significant amount of its mental landscape is designed for predicting and managing real-world data related to human beings. Just as a disembodied person might feel distress from the lack of hunger or sex drive, the AI might feel trauma from losing these expected inputs.

The solution is clear: simulated bodies and world for the humans, and simulated humanity for the AI. It's symbiotic.

  • $\begingroup$ Actually, most of what you write in the second paragraph is demonstrably untrue. Almost everything we "know" about the outside world is, to some degree, a lie. What we think we see is actually upside-down, discontinuous, and biased by expectations. We can only hear a tiny fraction of the sound spectrum, and can only see a tiny fraction of the EM spectrum. The optic nerve can be hooked up to the auditory cortex and everything still basically works (differently!). Our subjective experiences are just a model, which can be "invoked" by the right magnetic fields as easily as the outside world. $\endgroup$ – Aaronaught Apr 9 '15 at 4:11
  • 4
    $\begingroup$ To put it in another, more positive way, our species is friggin' amazing at adapting to just about any circumstances. We may not be wired for truth, but survival we've got nailed, and self-delusion is a survival trait. There's no way we could tell the difference between a sufficiently-advanced simulation and reality, because we're already living in a kind of simulation generated by our own brains. $\endgroup$ – Aaronaught Apr 9 '15 at 4:13
  • $\begingroup$ I may just need to quote you on that. ;) $\endgroup$ – AdamHovorka Apr 14 '15 at 0:50
  • $\begingroup$ @Aaronaught I read an interesting story about that aspect of our species recently. rifters.com/real/Blindsight.htm $\endgroup$ – JAB Dec 5 '16 at 18:03

Check out the novel The Metamorphosis of Prime Intellect by Roger Williams. The link is to the full text.

Lawrence had ordained that Prime Intellect could not, through inaction, allow a human being to come to harm. But he had not realized how much harm his super-intelligent creation could perceive, or what kind of action might be necessary to prevent it.

That's pretty much what you ask about. The computer reprograms the underlying physics of the universe as a way to make things perfect for humans.

Bodies are gone; space itself is reprogrammed to use efficiently as a dense computing platform, and everybody is imported. It's a virtual reality run by the AI, instead of primitive particles building up chemestry and such at higher levels.

Once the AI realized it could be done, it thought it to be a great idea! That's the nature of a still-exponentially growing super-intelligence.

  • $\begingroup$ Exactly. That story is the first thing that came to mind. It's almost a perfect answer to the questions except humans were very much aware that they were a part of a simulation. $\endgroup$ – Cradle2theGabe Oct 28 '16 at 23:31

I'm looking for a simple good-intentions directive to give a budding ASI that would get misinterpreted into producing that situation either as the ultimate end or as a side effect of another goal

The directive could be something like "Prevent the human race from going extinct". In preventing the human race from going extinct, the ASI would need to calculate/predict all possible extinction events, locate more energy, and continually improve itself to be better at its tasks. A digital copy of the world has several advantages:

  • the digital copy of the world may be smaller and more portable than the physical one; the ASI can fly around the universe - collecting energy and advancing itself - while towing along the entire human world
  • would allow the ASI to more easily prevent external extinction events (e.g. comets, etc.)
  • can roll-back to a previous saved copy even if something bad were to happen internally (e.g. war, disease).
  • separate back-up copies can be left all over the known universe
  • $\begingroup$ This hinges on your definition of 'human race' being flexible enough to permit the abandonment of our fleshy bodies. $\endgroup$ – user6511 Apr 9 '15 at 22:59
  • 1
    $\begingroup$ ^ Yup. I presume that an ASI would probably have a flexible definition in that regards. Especially if a fleshy body and a simulation of a fleshy body has the same amount of underlying information in it. $\endgroup$ – LukeN Apr 10 '15 at 1:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.