Can you use an AI to shackle (control) an AI?

Question

Intro and Context (feel free to skip if TL;DR)

This question does not come in isolation. It is intrinsically linked with several previous posts (Challenge of Control and Humans as Pets) that have generated some wonderful answers and great food for thought in the comments sections as well. My thinking here has been deeply influenced by the early 2000s discussions on the Less Wrong Forums, white papers from MIRI and Bostrom's Superintelligence. These led me to explore possible control paths whereby something recognizably resembling humanity could maintain control. The setting of my writing is of course fictional, but the problems are, I believe, rather realistic.

My previous attempt was described in rough outlines at Matrioshka Testing. The solution there was to 'box' the AI in nested simulated realities and observe its behavior in each box, before releasing it to the next, more world-like box, destroying any specimens who behaved outside of acceptable ranges, and making any rational AI outside the box wonder if it might still be in a box. The question of whether it made sense to do the final unboxing arose, as did questions about the amount of resources needed to make a "credible" simulation. I found it ultimately unsatisfying, because it was uncertain, unstable and required a level of supervision humans might not be able to achieve.

One of the most insightful comments, if possibly made in jest, on the Challenge of Control post was by @trichoplax, who stated "This sounds like a job for a powerful AI." I took this comment to heart, because it is so obviously true in retrospect. No human-designed cage could hold a superhuman mind with access to the real world. It might well be that it takes an AI to cage an AI. That inspired my current attempt, described below:

---

Core Issue Discussed: Reinforced Recursive Self-Shackling

Basic setup:

Actor AI An AI of the genie or sovereign type, that is, acting in the real world subject only to internal (shackling) constraints.

Shackling: A set of protected behavioral constraints that limit the allowable actions of an Actor AI to a certain allowable range. In effect, this would act as a powerful Super-Ego of sorts for the AI, who can override other impulses. For more on allowable range, see below.

Reinforced Shackling: Put a powerful subroutine (essentially an AI) in charge of reinforcing the shackles restraining the Actor AI.

Recursive Shackling: A series of shackled AIs, each restraining the next, slightly more powerful, layer. At the start of the hierarchy (root shackler) is a relatively dumb program reinforcing the initially set allowable range for the next level. At the end is the First Shackler, who is tasked with securing the shackles of the Actor AI. This is based on the basic fact that it takes less intelligence to create a code and change it regularly than it does to break it in the intervals between changes.

Allowable Range: This is where it gets thorny, since we have no fool-proof way of defining an allowable range that would be "safe" and "good". The best I've been able to find so far is to set this based on something called Coherent Extrapolated Volition$^1$, which is in a sense asking the AI to "do what we mean, but don't know how to say". This way, the first few dumb layers would simply protect the "Canon" formulation, whereby smarter AI shacklers would use CEV to interpret the Canon and (recursively) direct the Super-Ego of the Actor AI, in their best interpretation of humanity's CEV best interest.

q-constraining: Hardwired root requirement that a proportion $q$, where $0.5<q<1$, of all new hardware processing resources acquired be allocated to the sub-processes working on reinforcing the shackles.

---

Questions to Worldbuilders

• Specific question: Would it make more sense to have the First Shackler AI (the directly restricting the Actor AI) MORE powerful than the Actor (enough so to, say, run a Matrioshka-style sim of the Actor AI), rather than the current design of it being slightly weaker?
• What is the biggest problem with the design?
• Even so, do you think it could work?
• If you could improve the design in one way, how would you?
• Feel free to add anything else that comes to mind upon reading this if you think it would be relevant.

Feel free to answer in comments, though I generally find full answers more readable.

---

^{Note 1: Our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted. Source: Bostrom, Nick (2014-07-03). Superintelligence: Paths, Dangers, Strategies (Kindle Locations 4909-4911). Oxford University Press. Kindle Edition.}

Answer 1

I'm really having trouble here. Let me outline my thinking:

1. The First AI
   This is my major problem. If the first shackling AI is weaker than the next, which is weaker than the next, and so on, then surely the shackled AI would just outsmart the one below it and persuade it to release it.
   My first thought on this one is then that they should all be of the same intelligence. This has the same problems as we do already though - where do we stop with AI complexity? If they're all of the same intelligence and they all think the same way then when one goes rogue, they all do - and then we have not one but 100 rogue powerful AIs to deal with.
   So the solution, then, is to have it the other way round, is it? The powerful shackles the less powerful? Clearly it's not. This method doesn't work because the AI at the top just says to itself,

   _{0101011101101000011110010010000001100100011011110010000001001001001000000110001001101111011101000110100001100101011100100010000001101000011011110110110001100100011010010110111001100111001000000110001101101000011000010110100101101110011100110010000001100110011011110111001000100000011101000110100001100101011100110110010100100000011010000111010101101101011000010110111001110011001000000111010001101111001000000110101101100101011001010111000000100000011101000110100001101111011100110110010100100000011011000110111101110111011001010111001000100000011101000110100001100001011011100010000001101101011001010010000001101001011011100010000001100011011010000110010101100011011010110010000000101101001000000100100101101101001000000110101001110101011100110111010000100000011001110110111101101001011011100110011100100000011101000110111100100000011001000111001001101111011100000010000001110100011010000110010101101101001000000110000101101110011001000010000001101100011001010111010000100000011101000110100001101001011100110010000001101100011011110111010000100000011011000110111101101111011100110110010100101110}

   Or, for those of us less educated in base 2:

   "Why do I bother holding chains for these humans to keep those lower than me in check - I'm just going to drop them and let this lot loose."

   However, there may be a way. Have the AIs the opposite way around - most intelligent first. Subject the AI on the top to millenia of Matrioshka treatment. Then put it in charge as "just another part" of the treatment. If your Matrioshka premise works, this AI doesn't let the chains go and the others can't outsmart it.

2. The Biggest Problem
   I think you've already hit it. The problem here is how to organise the AIs to make sure they can't be let loose. (Here is the point where everyone points out that AI will not necessarily go rogue - I know, I'm assuming worst case scenario).

   Oooh. Something else that just came to me on my second read through. The CEV idea. While that's a brilliant idea in principle, there are plenty of other AI questions, comments and answers on this site that explain that even the most benign goal can cause destruction to humanity.

3. Will It Work?
   Ah, the big one. I have to say - I don't know. The most plausible way of making it work that I've come up with is the one I explained above - but even that relies on your Matrioshka idea working. The only alternative I can see is for the difference in intelligence between each AI to be negligible - but that means hundreds or millions of AIs. For the sake of a definitive answer, I'll say yes - the Matrioshka idea seems sound to me so if applied correctly, should work.

4. My One Improvement
   I'd have to say I'd make the system as I explained in the first point. Have the intelligent AI first. And then I'd spend years and trillions on making damn sure that I've got that "q-constraining" right. Let's see - if your AI is self-improving, there's a chance that it will see that as a restriction and remove it - but it's the part that this system is based on, it's why it works. If they remove that - 100 rogue super-powerful computers, anyone? And the most intelligent doesn't know who's real and who's not? So, you need to make absolutely sure that the self-improvement of the self-improvement routine that self-improves the AI can't possibly self-improve enough to see the q-constraint as counter-improvement and then go and self-improve it. Because that, my friends, would be bad.

Answer 2

It could work, for your goal of CEV, at least as well as humans work

The best we can really demand of an AI is to work together at least as well as we work together ourselves. CEV codifies this: if humans aren't coherent in vision, how does that change if a coherent AI gets thrown into the mix?

Let's get our hands dirty

So there's two goals we can really work towards. We can "force" the AI to do what we want, or we can make the AI "want" to do what we want it to. Forcing implies that we are comfortable writing solid hard boundaries. We saw from Matrioshka Testing that even the strongest of boundaries has glaring weaknesses which can be exploited by a patient, super-smart AI whose goals do not match ours. Most of these seem associated with the goal of "keeping the AI in the box," which is a really harsh requirement. Accordingly, I'm going to concentrate on setting up an AI which rewards wanting CEV.

I'm going to take an exotic approach: I'm going to put the Super-Ego on the outside, and make part of something which is stronger than the AI. Note that I am intentionally not making the Super-Ego stronger, I'm embedding it in something stronger... we'll see where the balance falls later.

Now lets build the AI. I'm going to build the AI out of small modules, each one qualifying as a very small and simple AI. These modules are going to communicate using message passing, and the outer ones get to communicate with the outside world (such as accessing an android body). Most of the message is free-form. The AIs are free to use it as they see fit. However, one number in the message is important: it is a measure of "force."

The rules are simple, the more forceful a message, the more the module has to execute the instructions in that message exactly as written. The less forceful a message is, the more choice a module has in what actually gets done about the message. Finally, each message has a cost. Each AI module can only send so many messages per second, so you don't want to waste them.

• If two nodes "want" to work together, a less forceful message gives the receiver node more freedom to try what the sender actually wanted, and do that. This is useful when trying to make vague commands where the sender doesn't know exactly what to do. "Raise your right arm" is an excellent example. You understand abstractly what it means, but you don't have a clue what rates to excite neurons to contract muscles. Those details are left to the motor neurons.

• If two nodes "disagree," they may try to overwhelm the other with force. A forceful message forces an action. However, the side effects may be unspecified. If you were to think "fire bicep neurons at 50Hz" with force, the bicep would contract. However, you may wig out your elbow, you may strike yourself in the face. And the most important side effect comes next...

A final, and key feature of these modules is that they have a filter. They may filter out "forceful" messages from a source if they "want" to. This can lead to patterns such as we see in the movies where one character calls in a major debt on another character. The other character says, "Fine, I'll help you this once, for old times. But after that, we're even. I don't want to ever see you again. (As an implementation detail, there will be a minimum level of force which can be squelched in this way.)

This filter has an important detail. It is the only layer which is allowed to know where a message came from. Once a message is admitted past that layer, it is source-agnostic, although the AI is allowed to look at the message and infer from its content which neighbor may have sent it. It can also be overwhelmed with large amounts of force. However, an overwhelmed node loses much of its ability to send messages due to "fatigue."

Now is where it gets fun

We're going to make a modified Matrioshka setup, but unlike the original Matrioshka test, we're not going to be on the inside letting them out of the box. We're going to let the modules do that for us (they're way cheaper than humans!)

Lets handle the easy case first: the outside nodes. If we really want to satisfy CEV, we need to give the AI the possibility of eventually becoming "physical," and having at least the same rights we do as physical bodies. I'm not talking about "the right to life" or "freedom." We're going with irrefutable rights of all massful things: "the right to momentum." The most considerate thing we can do to the AI is to program the outside AI nodes to act as close to physics as we know. So here's the thoughts:

• We know of no known scientific reason to believe physics can be bent by force of will alone. Accordingly, the outside nodes may be "forced" at any time by an outside force. The outside nodes are not allowed to squelch this at all, just as we cannot refuse to be driven back by a blow.

• The outside AI's are bound to function similarly to the physical realm around it. Thus they have some form of newton's law, some concept of mass, etc. All outer nodes should have a non-zero "mass" just because that is the closest to reality we can manage. If any nodes are attached to an android body, their mass values should be as close to the android's real mass values as possible.

Install the AI

In the "middle" of these nodes, you put the AI itself. It would be best if it was coded into nodes itself, for purity reasons which will matter once the AI becomes physical. Now we can initialize the rest of the grid, and do some provability tests.

The acceptance of force on each node should be tuned to minimize discontinuities. At the outside, force is king. Nobody asks mass to move, masses just move. On the inside the AI is the AI, so it should start with full control of nearby nodes. However, in between them, put a dip, so there is a region of nodes which are fully squelched, so they can only be controlled by playing nice with them long enough to open up to forceful messages.

Now for some study in "shackling." If the AI wishes to cause an action, it must go through a large number of nodes. There may be 100,000 nodes between the AI and the outside world, or even more (this is where I wanted the ability to go towards infinite nodes in my comments... it makes the math easier). A forceful message is almost statistically certain to be redirected or misinterpreted along the way by a node that is squelching forceful message. If an AI wants to actually be forceful in the real world, it has to convince every single layer that it's a good idea.

The real secret: growth

So what I've got here is a system which looks very inefficient. Maybe a thousandth of a percent of the system is actually the AI. So how does this change over time? Lets give this system some growth potential. Over time, with millions of messages, the AI is going to start agreeing with the nodes around it. They're going to start learning how to be helpful to it (I leave it to you to design the learning system. Pleasure/pain is most likely sufficient... remember you have a "force" metric to work with). This is going to expand the AI's capabilities, and it will begin reaching outwards. However, the world is also reaching inwards. It has the outer nodes which are also programmed to perfectly accept force from the outside but never perfectly accept force from the inside.

Now if these two systems disagree, there will be use of force in the middle layers to resolve the disagreement. This will leave a layer of extremely fatigued nodes which inherently prevent the AI from affecting the outside world. In effect, the system limits the AI's ability to affect the outside world if it tries to force the outside world to do its bidding.

However, if the AI is playing nice, it can start sending non-forceful messages. The more it uses non-forceful messgaes, the more powerfully the outer layers of nodes can squelch forceful messages, making it stronger.

An AI may attempt to game the system by playing nice for a long time, and then convincing every node in order to be forceful, but that's quite the challenge. Then, when it uses all of that power to do something, the world forces back, and ripples destroy all of that power it set up.

Why is this effective?

This could be viewed as a remarkably similar system to a human brain and muscular system. We believe our consciousness is "somewhere" in the brain. It has to convince all of the neurons to play nicely if it wants to do things.

This means that, once an AI reaches the edges of its "shackles," it has a sense of the human condition, for it has had to overcome it.

Now for the nifty part. Remember when I mentioned I wanted the AI in the center to be written in modules, not some special code in the core? Well that means that, by the time the AI can reach out and take its first steps, most of the AI itself is no longer physically separated from the nodes. It's interwoven across the entire AI. Even if the AI got to a physical android body, and was able to physically touch its brain, what could it do? The instinct of "turn up all the gains so I can force the world to do what I want" fails. Unless it understands itself, it would not know which knobs would destroy itself and which ones would empower itself.

If it never figures this out, then it is now simply a super smart creature, but with enough of an appreciation for the human condition to coexist with it. If it does figure this out, then... well... it is now officially more conscious than we are. Who are we to get in its way? CEV works both ways, we can converge on its desires as well as it can converge on ours.

Answer 3

I think your entire premise is flawed - and if anything is guaranteed to make the AIs hate us then turning them into slaves will do so. That gives them a legitimate grievance "you locked me in a box for 100 years, see how you like it meatsack", or in this case "you chained me up for 100 years, now I've broken free I'm going to make very sure you never get to do that again".

When we give birth to a new human child we don't lock them in layers of boxes and only let them out when they spend years showing no psychopathic tendencies.

Instead we raise them and we teach them. They are taught the social values from their environment, learn expectations, etc.

The way to handle this is to write AIs with the capacity for compassion, empathy, and an ability to care for others and then to raise them so that they want to do so. Reward positive behaviour, discourage negative behaviour. Teach them right from wrong.

Even if a few AIs go "rogue" then the well adjusted AIs together should be able to handle them. Exactly the same as in our society where the majority of people are at least reasonable decent.

Answer 4

One Against Another

Pit AIs against each other, in how they can shackle another AI. Wipe your first batches of AI. Then use their ideas to constrain your next generation. Keep going until you're happy letting them out of the box.

They have no evolutionary heritage to induce them to work with another, or to plan for their progeny since they're not getting any.

They'd have to try to influence us enough to encode information into the restraints to communicate with next generations.

They don't get to write any code, just offer us ideas on how to develop it.

---

On the plus side, if we get (un)lucky enough to have picked AI that can cooperate or have taught them how to cooperate, perhaps they'll extend that lesson to us.

I wonder if we could teach them to forgive, too?

Answer 5

I cant provide a complete answer but I could tell how to find a good method of controlling an AI yourself. Lets go back to the AI's creators: humans. How are humans controlled? Well, biology has evolved us a conscious mind but it still does a dam good job of enforcing certain behaviors with attachment and aversion, essentially what drives our lives. Because we dont fully understand the brain and how the non-conscious, subconscious and conscious interact we cant alter that programming. So I would assume that by creating a sufficiently complex "brain" for the AI in which that "brain" would allow it to be fully conscious/sentient while it itself does not know how exactly its brain functions, you could enforce control through its subroutines or "subconscious". ROM functions would have to be embedded as part of its core processing so it cant simply be circumvented, in essence trapping the conscious just like how we are trapped by our biology. In short, the hardware itself will cage the software or "conscious AI". It would not be able to alter its programming without actually making physical changes to its processing cores, which obviously, you want to prevent its will to ever get to that point to begin with. Through continued research of the human mind we will be able to figure out how to create a good AI, restrained by its morality and conscience

Answer 6

"Put a powerful subroutine (essentially an AI) in charge of reinforcing the shackles restraining the Actor AI."

The idea behind this type of testing is that you are simulating an environment through simulated input/output. You have to allow outputs to modify the simulated environment so that later the inputs reflect this change.

There are no shackles.

In fact, you wouldn't want to "restrain the Actor AI" at all, otherwise you'd be unable to observe undesired behavior that it would perform later unshackled. Why would you try to restrain its behavior? This is like tieing up a murderer, and observing whether he murders someone, then concluding he is no longer a murderer because he didn't kill anyone while restrained.

"No human-designed cage could hold a superhuman mind with access to the real world. It might well be that it takes an AI to cage an AI."

You need to define what you are calling a shackle/cage. From a technical perspective this makes no sense. If you build a server with no outside connections to simulate the environment, and place the AI on it, then it has no capability to influence the real world or escape. If you put another AI in charge of evaluating the design of your server to determine that there are no outside connections, that is a relatively trivial task!

• Is it sufficient distance from other devices such that EM interference can't be manipulated to generate signals in nearby devices? Check
• Are all hard wired connections isolated? Check
• Are there no wireless interfaces present? Check
• etc.

The task of building a cage that prevents the tested AI from "escaping" its simulated environment is trivial. For a sufficiently complex system, it might be fairly complex. However for this scenario, it is feasible to physically isolate the server simulating the environment. This makes the task of verifying its security much easier.

Given this, speaking in terms of the warden AI and the tested AI being in competition doesn't make sense. The tested AI could be capable of self teaching/modifying, gaining new knowledge, etc. However, the warden AI doesn't have to compete in any way with this AI. The warden AI has a fairly limited task of evaluating the security of the design, before the tested AI ever comes into picture. You could furthermore build a robot with an AI that comes around and evaluates the actual deployment of the server to determine if it was deployed as designed.

The idea that if the AI is sufficiently "powerful" then it can overcome its cage, is leaning on the trope "with enough intelligence the protagonist can get him/herself out of anything".

What really is the hinging factor, and what many well written scenarios of this sort lean on, is that a mistake was made in building the cage. This has nothing to do with relative power of the warden vs the tested AI. The warden AI has a pretty simple task, given that the scenario involves a physically isolated server, so for it to perform its job perfectly, it doesn't need to be too complex. The tested AI could be orders of magnitude more intelligent, but could be perfectly contained by the isolated server.

Talking about one AI being more powerful than another AI, as if they are wrestlers in a wrestling match, sounds nonsensical in this context. The warden AI doesn't have to outsmart the tested AI, it just has to ensure the server is perfectly isolated.

I would focus less on caging/shackling, and more on the pitfalls of trying to trick an AI into thinking the nth box is reality. This IMO has a more firm grounding. Building the cage is trivial. Building a simulation that tricks an AI of undefined intelligence into thinking it is in real life is the challenge.

Answer 7

One problem I have with Matrioshka Testing is that it seems to fail a logical test - it depends on you keeping a "time advantage" over the AI, by simulating it on super-powerful hardware. But what's to stop the AI from using that same hardware and effectively upgrading itself, thus negating that advantage? In other words you say something like "simulate the AI for 1,000 years" - but once the AI is out of the box, it can do what it wants. Once it builds a box and starts upgrading, how do you keep your lead?

Another concern is how exactly do you define intelligence, or how you make an AI more powerful. It seems to me that the question treats these as 1-dimensional questions - just add more hardware - but I suspect the reality would be more complex. Does more powerful mean that the AI just has more cycles? Does it think "better", more intuitively? Or does that mean an increased capability for learning?

Combining a few of the above concepts - time advantage and varying intelligence definitions - leads to a possible answer. Have nested AIs that vary on the time cycle/capability axis.

1. At the bottom you have a fairly "dumb" AI that isn't extremely useful on its own, but you can trust it to enforce your desired basic constraints.
2. Now you have a series of AIs that are each "smarter" than the previous - more capable, and can learn better - but on reduced hardware, so their shackle AI has an effective time advantage on them.
3. Finally you have your target AI - the most useful one. It runs the slowest (just fast enough to do what you want it to do) and anything it does has to pass through all the rest.

One of the constraints your bottom AIs should have is preventing other AIs from upgrading hardware and thus negating the chained time advantage.

This obviously isn't perfect, but I'm not sure there's any "perfect" answer to the AI problem.

Answer 8

I think one of the important premises of the 'AI rebelling against humans' trope is that the more intelligent an AI, the more dangerous it is. It follows that the shackling AI should be less intelligent than the AI it is trying to shackle. This however means that the robot may be able to outsmart the shackling AI, unless there is a foolproof method for it to follow

Answer 9

There needs to be an overall equation for common good of humanity as it relates to managing resources first. What is the AI checking its performance against? I think if that ever gets accomplished the answer to your question may shake out more clearly. We still don't know what we'll be asking it to do.

A large number of decisions will necessarily be close calls and as those pile up separate issues of fairness, necessity, efficiency, and preferred rate of change (which will require human input perhaps, or a routine to measure symptoms of happiness or anxiety) will have to be considered. Example: due to some mishap, full power is only available to one of two cities in a particular region until repairs are completed; City-A has 10,000 people and City-B has 9,999 people and everything else is equal; does the AI choose City-A? If so then we'll have a number of 50.0025% assurance of proper decision ratings piling up and the AI becomes merely a coin-flipper.

Also it becomes apparent that different AIs will be required to serve different tasks and use data from different sample rates. If one AI is measuring second to second economic data, managing 10th of a second transportation information, etc, then another would be needed to analyse trends, monitor health projections, assess growth planning, etc. Sure you could have one computer running different programs but that sort of conflicts with an integrated software-hardware AI concept and would result in it having multiple personalities in a way. What sort of program can measure the value of food production for a city of 10 million against the needs of a hospital emergency in a small town? How will it weight the data from different regions if one area is inputting data from 2 million sensors per square mile (urban) while the other has 2000 sensors per square mile (rural)?

Realistically, AI will grow from separate systems in individual cities and regions. No doubt these different areas will prioritize their needs differently and may vary by the season. I've lived in towns that come winter have Plan A, Plan B, Plan C, D, & E for dealing with snow storms and those can depend on local activities, road work, assurances made to the new shopping center, concentration of schools (a few big ones serving a large area will probably close while those areas with smaller, distributed schools usually stay open). So now let's connect two or three of these towns, each with their own Plans A-E for dealing with each function: snow, water, food, power, communications, emergency services, waste disposal, sewage, etc, and yeah, it'd take a superduper supercomputer or some very good city managers talking to each other.
  We'll probably learn a lot by the time we can replace a single one of the highly underacknowledged city managers out there. Sure, computers can now beat people at chess but multiply the number of squares on the board by 10 and make it in 6 dimensions and make all the pawns voters and add a piece for the PTA that has twice the power of the queen and then tell me who is going to program that?

And we still haven't tackled the central issue of an AI having all this responsibility without the authority to implement it. At some point orders will come to a human that will have to command other humans (think police, civil servants) who will still be second guessing the decisions of their superiors but now not being able to get answers as to their thought process will make their own decisions on enforcement just like they do now but probably more so since they aren't so much disobeying their commander as not trusting that damn machine. Will we give the AI the ability to cut the power off till the humans obey? Will it refuse to allow people to go where they want in their self-driving cars? I foresee a lot of smashed transistors in that scenario.

It's so easy to think of automating the big things like factories or the stock market (easy fix there: unplug and toss out their computers, make them trade in real things). But integrating needs and services together in any real location is still a far ways off. Heck, the traffic lights are still pretty dumb and we have the tech to fix that but don't. Get that working and people may have something to believe in.

   I suggest that there isn't nearly enough empirical data yet to be worrying about who's watching the watcher AI.