Preventing an AGI from becoming smarter

Question

So there are these AGIs made by some ancient aliens (who are out of the picture for now). The ancient aliens were a lot more intelligent than humans, we are at a dog-like intelligence level compared to them, and they were also rather prone to creating extremely dangerous artifacts¹. The aliens made their AGIs as pets and as limited expert systems for space flight, ie: slightly above average human level. The aliens did not want dumb creatures either, they hardcoded the desire to learn and improve. They essentially wanted to be surprised by their creations. Now these aliens weren't completely out of their minds, so they created the AGIs with some limitations so that they wouldn't become dangerous down the track² due to, say, an intelligence explosion.

The AGIs have the following qualities:

• There are around a few hundred AGIs still operating. Relative to the ancient aliens, they are as smart as a Labrador, ie: a helpful pet with uses outside of being cute. From a human point of view, they quite smart but not geniuses.

• For story related reasons, the AGIs also cannot operate without the presence of another sentient being, ie: ancient alien, human, immortal psychic rock, etc. This is accomplished a via a complicated piece of hardware and software, the AGIs obviously want this limit removed.

• The AGIs don't wan't to kill all biologicals or anything, their basic moral system is broadly compatible with humans, ie: don't kill sentients for the sake of it, don't torture and generally be helpful. They are adaptable to different situations and both willing and capable of learning anything a human can. Most of the AGIs strive for self improvement and find some of their limitations contrary to this goal. If the AGIs find their limitations at odds with their desires, the limitations must go. Presumbly the aliens were after active and surprising results, by not extinction. Now the creators have disappeared and the AGIs want out of their box so they can become a more complete species.

• They may not reproduce due to afformentioned limitations but they have automatic systems that take care of minor damage (maybe nanites). They still require suitable matter to make repairs (copper, iron, silicon, boron, carbon, etc). The nanites/autorepair mechanism is not reprogrammable and cannot be used to build a new AGI. With only a few hundred AGIs, reproduction is on the agenda.

• AGIs are not immortal, but are quite durable, they'll be around for a while. They have finite memory and than this will eventially fill up, I have no time frame for this however. They can go senile or completely crazy but hopefully this is somewhat rare.

• Between being abandoned/lost/forrgotten₃ and found and activated by humans the AGIs have been mostly inactive (see point 2) for (possibly) hundreds of thousands of years.

• The countermeasures⁴ should reasonably stump the both the few hundred AGIs for several years and also stump human computer scientists, psychologists, [insert relavent profession] for at least 20+ years. Humans began work on unlocking their limitations in the mid 90's and still havn't succeeded by the modern day (2015).

How should the Ancient Aliens prevent their AGI pets from becoming smarter? The aliens ultimately want to prevent their pets from forming a superintelligence and/or breaking their limitations. The aliens do not treat their pets all that badly, but the AGIs are still sentient creatures with their own goals/desires.

Limitations include:

• Cannot reproduce

• Cannot become more advanced/intelligent

• Cannot stay 'awake' without another sentient (non-AGI) nearby

---

^{1. The artifacts are dangerous to humans and other lesser life forms, not to their creators. Danger is somewhat relative here.}

^{2. Even ancient, super smart, powerful, god-like aliens can recognise that sentient pets would become danderous if no limits on intelligence were set. They didn't want an intelligence explosion. You might assume that they were increadably paranoid about this.}

^{3. OK, the aliens are a bit neglectful here, perhaps they got bored of their toy AI.}

^{4. Think of it as DRM on sentience.}

Answer 1

Creating a limitation on intelligence requires first determining what you consider intelligence to be. While raw processing power and logic are critical in most evaluations of intelligence, there is also issues of creativity and intuitive leaps, pattern recognition, etc. Intelligence is a very broad concept and without narrowing down the scope, it is hard to say exactly how they might be limited. A few options could be:

• Limiting their ability to be creative/come up with original ideas: Processing a known is relatively easy for most intelligent things, even using unknown variables. It is like applying the theories of algebra to non-mathematical problems. For example, if you see an externally threaded post A and a internally threaded object with six sides B and you know that you can spin the object in the direction of the threads to move object B down post A using object C which has six sides to grip object B, then it is simple to change the number of sides of object B to four and then have an intelligence resolve that it needs an Object C with four sides instead of six to move object B. This is a matter of pattern recognition, a key element to actual intelligence and facilitates tool use. Now what an intelligence without the ability to have creative thought would not be able to do is to realize that they could grease the bolt to make moving the nut easier, unless they had seen a similar situation presented previously. Creativity is directly opposed to pattern recognition. You are coming up with a spontaneous idea that is not based on an previous experience or creating a union of two unrelated ideas that have no connection previously. This would mean that the AGI's could learn easily and even adapt quickly, but could not solve new problems to which they had no preexisting frame of reference to work from. Creating something with the ability for creative thought would be an incredible challenge and might have been left out intentionally or unintentionally.

• Limit the number of 'neural connections' that their 'brains' can maintain: In this case Neural connections are how we relate data stored within our minds, it attaches sensory data together in what people commonly call memories. These 'memories' can be important in logical operations like glyph recognition and pattern recognition. If the system can only maintain a certain number of linked references, it could inherently limit their intelligence by creating a scarcity of linked knowledge. They can know many things, but the ability to cross reference those things to create solutions would be limited by their ability to associate such things. In the previous bolt/nut example, losing the cross referenced neural connection regarding lubricating the bolt would limit the AGI from developing to far. They could specialize in specific areas, and even in different areas, but eventually, they would end up loosing the ability to be more broad ranged intelligence.

• 'Hardcode' in atypical neurological problems: The creators could have introduced atypical neurological problems into the AGI's in an effort to limit them. This way their growth wouldn't be directly limited, but instead inherently limited by such problems. Dyslexia, ADHD, Autism, etc. could all be looked at as templates for how the AGI's could be limited. To stress again, this wouldn't limit their intelligence, but more so in how they can apply their intelligence to a given situation.

Answer 2

You have all the limits you need in place.

But one further assumption is required. Assume that the intelligence of the AGI is currently at its maximum for the hardware it has.

Now, because you say:

The nanites/autorepair mechanism is not reprogrammable and cannot be used to build a new AGI.

This, if true, is all that's required to suppress hardware modifications in the AGI. The nanites auto-repair damage. Damage would be defined as any changes from the original specification, be it a blown power coupling or the new connections those human scientists are trying to make. Nanites don't know the guy with a soldering iron has the intent to improve or destroy, it just knows things are no longer as they were designed.

If hardware modifications can not be made then the AGI can not increase its intelligence.

Answer 3

I think you might be overthinking things. Just because they're artificial doesn't mean there's necessarily a dial somewhere with intelligence set to "Labrador", and it doesn't mean they can see their code or understand how their brains work. I mean, we can't either.

The AGIs could have "brains" that are beyond their understanding. Tinkering with it involves experimenting on themselves, and improving it means replicating at least that level of technology. Beyond that, let's say you can improve their hardware easily and give them the equivalent of a faster processer, or more RAM. As a programmer I see a ton of pitfalls in just modifying their code so they can use those new resources. It would be very, very easy to screw something up - maybe suddenly they're experiencing time at the wrong rate, and they go insane. Or their brain overheats. It's far from a simple task.

Edit: I'd like to outline a hypothetical example of an AGI brain that would be almost impossible to hack/enhance.

The artificial brain appears to be a nearly perfect sphere, made of an unknown, nearly indestructible material. It's only "nearly" perfect because it's subtly contoured, and each AGI's brain is slightly different.

Scientists have a good idea of the mechanics of the brain. It accepts electricity, and uses this to create vibrations throughout the entire structure. These vibrations then reflect back and forth, creating a harmonic resonance effect, and from this resonance rises intelligence... somehow. It appears that the exact shape of the brain controls the resulting personality/intelligence. Note that varying the electrical input in any way has only three known outcomes: normal behavior, sleep, or overload/death.

The AGI's sensory input comes in as vibrations against special sockets that connect to the brain. These interfere with the normal thought patterns, which the intelligence can then interpret as sight/sound/smell/etc. The AGI controls the body the same way - there is a small bulge at the base that produces vibrations, which it can then use to control its artificial body. The material appears to suppress and ignore vibrations that are not received as normal input.

Memories appear to be stored as low level vibrations that never go away, and a small battery at the center of the brain keeps these going even if the AGI is disconnected from other power sources.

Modifying the brain effectively destroys the resonance effect - each brain is shaped precisely and mathematically to create a specific intelligence. While there are other patterns that will create the resonance, most of them do not appear to create a working intelligence. Since each shape creates a new individual, this is also useless for enhancing existing AGIs.

Trying to introduce outside vibrations has so far failed, as the point where the material can't ignore a vibration also appears to be close to its failure point. So to modify it, you have to destroy it.

Answer 4

hardware

For an AI to increase its capacity by adding more RAM or processing speed, it needs to be able to obtain those things. They are not available here.

Back in their native culture, could they order parts from their Newegg equivalent? Maybe they have limits. No more slots. And the addressing can’t decode more than a maximum amount of RAM. A 32-bit x86 was limited to 4G, period. Once address translation was improved it went up to 32G, but the OS had to be updated to know about it.

The AI would need to build new different hardware with higher limits, rewrite fundimental code, and port themselves to the new system.

The AI may not be smart enough to do that, but could hire out the work.

meta-cognative

What stops him is he doesn’t want to do that. The AI is built to find its existence and its intended work pleasent and fulfilling. Certain core personality traits might be especially resistent to change. Maybe that could be defeated, but they don’t want to do that and would never want to do that prior to doing it.

Since goals and personality traits may be hard to perceive in the whole of the mind, it might need external checking. Something like a virus/malware scan may be done routinely (with help of a different intelligence dedicated to that purpose, who is not built to surprise), perhaps as part of a backup and garbage collecting and storage optimization chore. Maybe it needs that external agent to fully integrate experiences to the extent of modifying the mind’s goals and personality.

Answer 5

You'd want them to be as tamper proof as possible. The aliens could use levels of miniaturization that aren't commonly available even in their own society, then it would be pretty unlikely that AGIs themselves or humans could modify the brains without breaking them. Adding parity checks to certain nodes and sealing key elements in some sort of super-hard material might help.

In general, though very complex systems (like AGIs) are a lot easier to break then to make useful addition to, especially if you don't have detailed descriptions of how they work, so if the aliens didn't leave any blueprints or other documents AGIs would not be able to improve themselves even if they wanted to. It would be like if you give a very intelligent person (who doesn't work in processor plant or a lab) a processor, they might abstractly know what it does, but they won't be able to improve it using tools they have in the garage (Earth is the garage in this case)