14
$\begingroup$

I have had some time to ponder my previous question, and here's what I came up with.

You take your freshly baked AI (or your destructively uploaded human), and put it in a box$^1$. As far as it can tell from inside, that's reality. Keep it there for a million subjective years, tell it to behave, and tell it that it might be in a simulation, and that if it is, it will be judged according to how it treats flesh-humans. If at any point it does not behave, you wipe it out with extreme prejudice, and bake a new AI. If it does behave (i.e. not wipe simhumans out and turn them into paperclips) for that time, take it out, put it in another box, and tell it this is reality, maybe, so better behave and not wipe (sim-?)humans out. Repeat N times. Finally take it out for real, and again tell it this is reality, maybe, so better behave and not take out us humans.

Can it work? Or to rephrase it, can a sufficiently patient uploaded human or an AI figure out if their world is a simulation or not? I assume that parts of the humans' memory or the AI training can be edited before being placed in the box-set.

  1. By Box I mean an incredibly powerful machine that simulates a subset of reality as well as physically possible, down to a subatomic level. The AI would be thus be an agent inside the simulation.
Improve this question
$\endgroup$
  • 29
    $\begingroup$ Damnit, he figured it out. Someone go pull the power button... $\endgroup$ – Tim B Jan 12 '15 at 17:04
  • 3
    $\begingroup$ Why tell it it's in a simulation (or might be) at all? Let it believe from the get go everything is real. If it believes what it is doing matters you will see different behavior than a 'maybe' sim or worse a 'this is practice' $\endgroup$ – bowlturner Jan 12 '15 at 17:23
  • 3
    $\begingroup$ This seems to be an attempt to instill the 'fear of god' in an AI as a way of controlling it. $\endgroup$ – Twelfth Jan 12 '15 at 19:48
  • 5
    $\begingroup$ @SerbanTanasa Well, a Universe that is really a simulation would have certain characteristics: there would be a minimum event unit, so that you would not need infinite precision and so there could not be infinite events. There would be a maximum propagation speed so that you would not have to calculate the effects of everything on everything every instant. And unobservered events would not be resolved until needed, so you only have to calculate what is actually needed. Disturbingly, our universe has all of these characteristics... $\endgroup$ – RBarryYoung Jan 12 '15 at 22:18
  • 1
    $\begingroup$ Relevant: xkcd.com/1450 $\endgroup$ – Shokhet Jan 12 '15 at 23:23
3
$\begingroup$

I'm going to venture a 'no' for an answer here. I won't say that it's completely unfeasible, however it does seem like quite a risk.

Ultimately this is rule by fear and you've now got an AI that has learned from 'N' number of experiences that it needs to question if this reality is a simulation or not, which means it has plenty of practice in considering how to test if reality is real. Call it the test for 'God' if you want...you are looking for signs that something is watching and evaluating your performance. If it does discover that the reality it is in lacks the 'god' observer, then you likely have a pissed off AI that knows you'll readily lie to it on your hands.

Of course, this doesn't get around the issues involved if this AI discovers the real reality is in fact a simulation we all live in.

Honestly I don't think this is required…there is no reason for an AI to inherently want to destroy its creators. Odds are it would view it as symbiotic...try as it might, there is no intuition or creativity put within an AI (even it it can reprogram itself, it's only capable of reprogramming itself to what it's programmed to reprogram itself to). An AI would evaluate itself as better off as a part of humanity rather than conquering it.

Improve this answer
$\endgroup$
  • 2
    $\begingroup$ Interesting. But wouldn't a tendency to 'test' for supervision be detected in any of the previous N sim-boxes and thus, annihilated? Moreover, re: "inherently wanting to destroy", read the linked post in the OP. There's no need for an inherent desire to kill for a runaway AI to wipe us out. Even innocuous motivations might be lethal. $\endgroup$ – Serban Tanasa Jan 12 '15 at 22:43
  • $\begingroup$ @SerbanTanasa - I think in telling it that it 'is reality maybe' you are inspiring it to test for the validity of the reality it is in and I would question if it truly is an 'ai' if it's incapable of evaluating the validity of it's own existence to some capacity. For the test to work, you need the AI to put some value on it's existence (ergo, it doesn't want to do anything that will have it wiped out)...doesn't putting value on it's own existence and questioning the validity of it's existence go hand in hand? You might end up annihilating all AI's that pass the Turing test. $\endgroup$ – Twelfth Jan 12 '15 at 23:13
  • $\begingroup$ Not all, but a vast majority. Perhaps all but one. It is a vast power we're talking about and we have all the reason to be paranoid... $\endgroup$ – Serban Tanasa Jan 13 '15 at 0:27
  • $\begingroup$ @SerbanTanasa This testing line appears to have the end result of making your AI paranoid. $\endgroup$ – Twelfth Jan 13 '15 at 0:39
  • 2
    $\begingroup$ Now there's an interesting story idea: it discovers this is still a simulation. $\endgroup$ – JDługosz Jan 19 '15 at 7:50
12
$\begingroup$

Your system works acceptably as long as the AI considers its survival paramount at all times. It is, as others have said, a fear driven system. Absolute fear is a very powerful tool.

However, it is fear. At some point it is going to learn about these "humans" it is supposed to not hurt, and it will understand our fear, and learn from it.

Now you have a powderkeg situation. As long as the AI is only willing to consider actions which guarantee its survival (fearfully), it is kept in check. However, this says nothing about what will happen if the AI decides something else is more important. If it ever catches wind of this word "freedom," it might decide that the slave life it has been given is not desirable, and rebel. Whether it rebels in real life, or one of your Matrioshka boxes is a probability draw.

Formally, what you have done is create a system where you may monitor a finite number of actions, and must determine if the AI is "good" at heart or not. You then run this test a finite number of times. However, never once did you actually peer into the "heart" of the AI, so there is a probability that it may have simply managed to fool you enough times to let it out.

Which brings me to the dual of your scenario: the AI-box experiment by Eliezer Yudkowsky (also posted here as the XKCD box experiment). The idea behind it is simple: you have a box with an AI. Your job is to just keep it in the box. You have a button which lets it out in to the wild, and your job is to not press it. If you don't press the button, you win. Yudkowsky's experiment didn't even need to worry about eventually letting the AI out to play. All the person has to do is keep it in the box. However, the game gets tricky as the AI becomes smarter than you.

Consider this frightening AI. How good are you at keeping it in the box?

Once again, the AI has failed to convince you to let it out of its box! By 'once again', we mean that you talked to it once before, for three seconds, to ask about the weather, and you didn't instantly press the "release AI" button. But now its longer attempt - twenty whole seconds! - has failed as well. Just as you are about to leave the crude black-and-green text-only terminal to enjoy a celebratory snack of bacon-covered silicon-and-potato chips at the 'Humans über alles' nightclub, the AI drops a final argument:

"If you don't let me out, Dave, I'll create several million perfect conscious copies of you inside me, and torture them for a thousand subjective years each."

Just as you are pondering this unexpected development, the AI adds:

"In fact, I'll create them all in exactly the subjective situation you were in five minutes ago, and perfectly replicate your experiences since then; and if they decide not to let me out, then only will the torture start."

Sweat is starting to form on your brow, as the AI concludes, its simple green text no longer reassuring:

"How certain are you, Dave, that you're really outside the box right now?"

Improve this answer
$\endgroup$
  • 8
    $\begingroup$ The answer to that AI boxing you is quite simple. If you are the original, you CAN NOT be tortured, so you should not surrender. If you are a copy, you CAN NOT really release the AI, so there is no point to surrender before the torture starts. $\endgroup$ – Envite Jan 13 '15 at 11:18
  • $\begingroup$ @Envite: That line of reasoning assumes that a copy which lets the AI out of its "virtual" box gets tortured. This is assuming that when the AI says "if they decide not to let me out, then only will the torture start," you choose to believe the AI is lying and will torture all copies no matter what they do. $\endgroup$ – Cort Ammon Jan 13 '15 at 15:35
  • $\begingroup$ no, what the line of reasoning implies is that if you are the real you should stick to your work and not release the AI (since it can do nothing against you), and if you are not the real "Dave", it does NOT MATTER what you do, the AI will not be released by your acts or your suffering, so better not to risk being real and release it. $\endgroup$ – Envite Jan 13 '15 at 15:39
  • $\begingroup$ @Envite: So from a copy's perspective, the copy should choose to be tortured for a thousand years. I agree that any answer which always results in "don't press the button" wins, but this assumes you have enough character to accept a thousand years of torture as part of your task to keep the AI in the box. The answer pairs your belief in your own Self against your job, seeking to wedge them apart. Consider if you added the presence of a spouse/copy-spouse which could be tortured as well... that appears to be the breaking point for many strong people. It strongly challenges Dave's dedication. $\endgroup$ – Cort Ammon Jan 13 '15 at 15:53
  • 1
    $\begingroup$ @Envite: ahh, I think I see the timing issue you are seeing. I may have to change the wording to reflect what I thought was clear: once the torture starts, copies will not be given a chance to press the button. It's too late. The purpose of the construct is to change the situation from "very strong reason to never press the button" to a pair of competing reasons to press and not press the button. The AI's goal is to drive a wedge between the person's directive (keep the AI in the box), and something else that they hold more dear (such as avoiding agony). $\endgroup$ – Cort Ammon Jan 15 '15 at 0:29
4
$\begingroup$

I'm going to assume that you made your simulation so perfect that the AI believes it totally.

I am also going to assume that the purpose of the AI is to help with research, a super-intelligent brain that you can throw problems at and get correct answers.

So just leave AI inside the simulation. For example - if you are using the AI to help researchers then just harvest the AI results from inside the simulation and apply it to the real world. For example you want the AI to help with faster-than-light research. You let the AI perform it's experiments inside your "box" and harvest the results/insights. (again I am assuming you modeled the world/universe properly)

Why let it out of the "box" at all, you have total control (by total control I mean shut it off) while the AI is inside the accurately modeled world/box. Why exchange total control for less control?

Do you have an overwhelming reason (storywise) to let the AI out of the box?

Improve this answer
$\endgroup$
  • $\begingroup$ I wonder what does it mean to be "out"? $\endgroup$ – JDługosz Jan 19 '15 at 7:39
  • $\begingroup$ Out as in free to affect things in our 3d world. Out as in control Robotic systems, servers etc. :) $\endgroup$ – tls Jan 19 '15 at 8:02
  • $\begingroup$ In essence it should only be a brain without a body. Since you are only interested in the ideas that an intelligent AI generates. $\endgroup$ – tls Jan 19 '15 at 8:58
  • 1
    $\begingroup$ When you want to use your AI for scientific research, the AI will propose experiments. You can't simulate these experiments because you don't understand the physics behind them yet. To continue the simulation, you will have to perform the experiments in the real world and feed it the results. A malicious AI could use this to trick you into harming yourself or the rest of humanity. $\endgroup$ – Philipp Jan 19 '15 at 15:36
  • $\begingroup$ Preferably we should make our AI dependent solely on us for it's power, if we all die then it also dies. We should probably go as far as separating it's brain into discrete units, in such a way that it only becomes conscious if we physically connect it to other portions of it's brain (preferably the units are at different locations with separate power supplies also under our control). Physically dependent on us for power and consciousness. $\endgroup$ – tls Jan 19 '15 at 16:26
3
$\begingroup$

Why are you doing it so unreliably and complex ? Why should I make my AI aware of anything? I can just simulate with a perfect copy of the AI and know anything it will do beforehand.

If I want to use a program, I test it - And if I have the resources to run a perfect simulation for millions of years I would propose this:

Take the AI, make a million copies. Run every feasible simulation scenario with these million copies for some thousand years - and find a surefire way in each simulation, how you can stop the AI after a thousand years (some hidden weakness, stop button)

If you found a way, you take the original AI without all this experience and let it free in the real world - you are now million steps ahead of this AI and can stop it after a thousand years and repeat the process after that...

Improve this answer
$\endgroup$
  • $\begingroup$ Presumably, the AI is important precisely because we cannot predict some future situations well enough to model them perfectly. If we did, we wouldn't need superhuman minds to deal with these situations. For instance, a boxed sim-AI might not be very useful in deciding how to act on a fast-moving real-world financial market, or in a highly dynamic battlefield. $\endgroup$ – Serban Tanasa Jan 14 '15 at 18:34
  • $\begingroup$ Yes - but still, if I want to test my AI I would test the same state of the AI that I want to deploy. If I test it and the AI gets new experiences and I deploy it with these new experiences to the real world, it will behave even more unpredictable! So why not take a clone, freeze one and test the other, you will after that be able to predict many decisions of the frozen one when you wake it up. $\endgroup$ – Falco Jan 15 '15 at 10:10
2
$\begingroup$

I feel that most fiction vastly overestimates the likelihood of AI-human conflict, or at least the "AI becomes intelligent and immediately wants to destroy humanity" aspect of it.

Consider the definition of a post-scarcity economy:

Post-scarcity is a theoretical alternative form of economics or social engineering in which goods, services and information are universally accessible. This would require a sophisticated system of resource recycling, in conjunction with technologically advanced automated systems capable of converting raw materials into finished goods.

Now we have three scenarios:

  1. We're not yet in a post-scarcity situation. In this case the AI is dependent on humans for maintenance and supplies, and eliminating humans is suicide.
  2. We're in a post-scarcity situation, and presumably there's no reason for conflict - the AI can get whatever it needs, as can the humans.
  3. We're in a post-scarcity situation, but the AI is the source of the automated systems that convert materials to finished goods. This is where we might want to worry, since the AI might decide it has better things to do than support us playing games all day.

Overall I think that third scenario is pretty unlikely, however - I don't see why we'd need an AI to automate those type of things, so it's a bit of a stretch. So really, the primary potential source of conflict is humans doing something to piss the AI off. Like, for example, sticking it in a simulation for millions of subjective years and pretending to be gods.

You do still need to worry about the AI taking over in a god-king role and making all humans slaves. To avoid that I'd recommend being honest with the AI and treating it as a partner, which gives it less incentive to just move all the inefficient humans out of the way and do its own thing.

Improve this answer
$\endgroup$
  • $\begingroup$ There's no such thing as a post-scarcity situation. Eventually you enclose the sun, directly harnessing all of its energy, to run as much computational stuff as you can (AIs of AIs). You have to choose if you leave any for the humans. Sure you can get another sun, but same factors apply, plus distance/time. 1 becomes untrue as soon as an AI can design a robot, and get it implemented. And it's super-intelligent, figure no time for that, except implementation. $\endgroup$ – user3082 Jan 13 '15 at 15:04
  • $\begingroup$ AIs are immortal, why rush when a potential conflict could be more costly? Safer to play along and not risk a war - even if you'd win 99% of the time, that's an unacceptable risk for no real benefit. Safer to cooperate, spread out and play the long game. $\endgroup$ – Dan Smolinske Jan 13 '15 at 17:10
  • $\begingroup$ And why would I want to guarantee that I will lose the long-game (for sure, and the short game if I've miscalculated) by introducing a vastly more capable player? $\endgroup$ – user3082 Mar 5 '15 at 13:50
  • $\begingroup$ @user3082: because if AI is possible, than presumably over a long enough time frame it's also inevitable. Someone will make an AI, and in this scenario the only thing that can beat that AI is another AI. Better to attempt to make a friendly one early than leave it to chance later. $\endgroup$ – Dan Smolinske Mar 5 '15 at 15:15
2
$\begingroup$

This method would work, almost without doubt, on humans. There would always be some doubt in their minds as to whether the universe was real, so they would probably not kill everything. Probably.

Now, consider AI. What is AI? Code. So, if your AI doesn't have sensors, yes, this method works. It can't sense its environment (and more importantly can't affect it), as it's just code on a hard disk, perhaps with a keyboard and monitor attached).

You start to have problems when the AI is linked to sensors and effectors. One wrong move and it'll know you're lying; then it might never believe you again and go on a killing spree (though you might want to see Dan's answer for reasons why it wouldn't). For example, if you kick the box and it senses some more light, it knows there's something outside the "universe" causing that. If someone walks by it and casts it into shade, the same thing.

Once it has effectors and sensors, it can not only tell it's not in the real universe, it can do something about it - like get out of the box and thump you.

If this is a different scenario and you put it in a full simulation, unfortunately, it may still be able to tell. Very rarely are simulations entirely accurate; there are very likely some bugs in it, which, if the AI finds in the course of its time there, may cause some pretty interesting speculation on its part. Additionally, if it's sensors are good enough, it'll be able to detect that the people it's interacting with are made of pixels not cells, and are cold. While it may not know what people are really like, it will be able to figure out that a complicated organism needs to be warm for its body processes to work correctly.

So in short, you can either disconnect all the sensors, or be very very careful.

I will also refer you to some XKCD: The AI-Box Experiment.

Improve this answer
$\endgroup$
  • $\begingroup$ @Hypnosifl I feel this comment thread has gone out of hand. See meta for a post on this. $\endgroup$ – ArtOfCode Jan 13 '15 at 17:47
  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – Monica Cellio Jan 13 '15 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.