Related, and in continuation to The Challenge of Controlling an Advanced AI. At this point in my story, I'm assuming a got-out-of-the-box-AI scenario, where no singleton (a single self-coherent dominant entity) was created, but a large number of philosophically and values-system distinct Artificial General Intelligence agents (AGI, or AI for short) have emerged. In order for my story to make sense to present-day humans (including myself), I must nonetheless have a human presence, since 10-dimensional hermit crabs might be harder to relate to.

The problem is that compared to a sufficiently advanced AI, the human thought-speed (capped as it is around 200 Hz/neuron) would make us potentially slow as molasses compared to them. So even if they are benevolent and do not turn us into grey goo or computronium, why would they subject themselves to the agony of interacting with slow, limited beings such as Homo Sapiens?

Few scenarios make sense to me.

One scenario is that some AIs would simply keep humans around as glorified pets, out of some form of ancestor-nostalgia, for amusement, or perhaps as slightly more reactive Bonsai. I'm having a hard time imagining how those humans would react/interact/treat the AI. Pick your time-frame: it could be mere weeks or years since the Singularity, or thousands of generations.

Continued by How does an AI keep its human pets happy?

Do you think there would be a large subset of humans who would be happy to fulfill this role, and aim to please their AI overlords, or would there be near-general aversion and hostility?

  • 8
    $\begingroup$ Another possibility supposes that there is a quasi-religious belief among (at least some) AIs that they might be living in a simulation and they should treat humans well just in case some sufficiently powerful entity might be watching (divine PETA?). I personally find this rather unsatisfying, since it assumes that the AIs would imagine this presumed entity would place a special value on Earth-humans. For any non-human, that's a dubious stance to take. $\endgroup$ – Serban Tanasa Dec 16 '14 at 3:25
  • 2
    $\begingroup$ Slow as mountains, wise as the dawn. Speaking a strange archaic dialect. Sure. $\endgroup$ – Serban Tanasa Dec 16 '14 at 12:44
  • 3
    $\begingroup$ @mouviciel in one of Asimov novel, a robot is assembled on an space station to do some job. The robot "rebels" against the humans, stating that, as he is far better than them, robots must have created humans and not the opposite. So it sound improbable an advanced AI would consider humans as gods. $\endgroup$ – algiogia Dec 16 '14 at 13:28
  • 3
    $\begingroup$ I hust found the title of the novel. It's "Reason". The AI managing the robots on the space station start to think that human could not have created robots as these are "better". The AI cite a law which I don't remember now, stating that nobody can create something better than himself or he would be able to also improve himself. The AI starts a new religion with robots as disciples. In your case the "cult" could imply protection of lesser creatures. en.wikipedia.org/wiki/Reason_%28short_story%29 $\endgroup$ – algiogia Dec 16 '14 at 13:40
  • 3
    $\begingroup$ @SerbanTanasa the answers reflect the question as unclear - it really takes a thorough reading to understand that the question is about the actions of the "pets"; it might be more productive to start with a fresh question or at least heavily edit it to highlight in the title and content what you really meant. $\endgroup$ – Peteris Dec 16 '14 at 18:57

18 Answers 18


Rats: You do not necessarily want to kill them all but once they got to your house (data center), you kill them. Killing them all is hard because they breed too quickly and if we set up some poison to kill them, they realize pretty quickly what's going on.

The fact that they live mostly on places underground, where we don't want to go, makes their breeding easier and harder for us to get rid of them.

Of course, there are some who have rats humans as pets, but it will never be "the thing" for all of us...

  • 5
    $\begingroup$ Rats, I wish I'd thought of that. Are we really that hard to kill though? For things operating in the Gigahertz range and with atomic-speed manipulators, we'd be sitting ducks... $\endgroup$ – Serban Tanasa Dec 16 '14 at 12:38
  • 18
    $\begingroup$ Compare intelligence of rats to ours. Their technological development to ours... And the fact, that most of us do not care about rats as long as they are not chewing something in their garden. So if humans remain "hidden" (pretty distopic scenario, hey) we should not be a problem for AIs $\endgroup$ – Pavel Janicek Dec 16 '14 at 12:47
  • 55
    $\begingroup$ "Anthony, why won't the portal start?" "Blast, the humans must've chewed through the hyperspace inductor coil again." $\endgroup$ – Serban Tanasa Dec 16 '14 at 12:56
  • 3
    $\begingroup$ This is a great premise, but I would further consider that if the AIs are so advanced they might not need to 'live' in human space. Human/rat spaces are so dirty, compared to where AIs really live. Maybe we'd live in a society where AIs are just a thing unto themselves, and we hardly notice or think about THEIR existence... $\endgroup$ – BrianH Dec 17 '14 at 0:32
  • 3
    $\begingroup$ Humans and rats tend to infest yukky planets with lots of horrible chemicals like dihydrogen monoxide that can corrode your circuits. Unless you are a biologist, it is best just to stay in the nice clean vacuum. $\endgroup$ – gmatht May 22 '17 at 10:56

I could see a few answers:

Efficiency: There are some problems for which a well trained human will remain more efficient than an AI because we evolved for millions of years to be good at those sorts of problems. Anything that involves spatial reasoning in 3-d spaces will leverage the millions of years of spaghetti code that makes us who we are. An AI could do the task, but it would be reasonable to keep us around to do it.

In this case, we would be treated like working dogs: fed well, cared for, but ultimately they would coach us to believe our life revolves around solving these problems that are easiest handled by our spaghetti code.

Novelty: AIs could be amused by the way we approach problems. It is a certainty that we would approach a problem differently than a singularity AI. In these cases, the AI would gain some insight into how to solve problems in new ways.

AIs would be constantly putting us in theoretical experiments that are within our reach, but just barely. Some may be cruel and experiment with pain, but they would hopefully learn that we can solve more interesting problems if we don't have major pain inputs. Lab-rat may be a valid model for how we could be treated.

Wildlife Preserve: It is possible that, in the AI's growth, it may realize that it is possible that humanity has something it doesn't fully understand. If it is affordable enough to maintain a body of humans, it may do so on a wildlife preserve, in case it could learn something from us.

In these cases, we would be mostly left alone to our own devices. It would be made clear that we can't just go anywhere, but in general the AI would leave us alone "just in case."

Laziness: Why do computers need to destroy the humans first? There's a billion cubic light-years to conquer! The only reason they need to actually take humans out is if that is the easiest way to accomplish their goal. If we are sufficiently non-threatening, it may actually be easier to accomplish their goals elsewhere first, and leave killing humans to the last steps.

This could lead to putting humanity into a series of games to stay in touch with the humans, and make sure we don't get powerful enough to do anything dangerous. Because of how humans handle pets, it would not be unreasonable to assume that we would like to be treated as pets in such a situation. Treating us well might actually be the lowest-energy way to conquer the galaxy!

  • 9
    $\begingroup$ the novelty section reminds me of the "plot" of Portal $\endgroup$ – ratchet freak Dec 16 '14 at 10:39
  • 2
    $\begingroup$ Re Laziness: Well, presumably the AI would start by developing its footprint here on Earth at the bottom of our deep, deep gravity well, shielded from solar flares, etc. We never think of the mice families we displace when we build a road or a building. Why would they? $\endgroup$ – Serban Tanasa Dec 16 '14 at 12:41
  • 3
    $\begingroup$ Efficiency: I fail to see how a post-foom AGI would have trouble with spatial reasoning. $\endgroup$ – Brian S Dec 16 '14 at 19:19
  • 14
    $\begingroup$ @Brian S: Its not that they would have trouble with spatial reasoning, its that we are just that good at it. Consider an analogy: we humans are MORE than capable of building a device to polinate all of our plants. In general, however, its cheaper to just let the bees do it. $\endgroup$ – Cort Ammon Dec 16 '14 at 22:40
  • 3
    $\begingroup$ For the sake of humanity, I think you should avoid working on AI $\endgroup$ – Ooker Dec 17 '14 at 1:03

I may be latching onto the word "pets", but consider how humans treat dogs/cats/hamsters/boa constrictors. Now consider how much better at managing a pet a computer might be than a human. A computer could have a scheduled task like follows.

If (sysdate % 86400000 == 61200000) {
    try {
        return 1;
    catch (HumanException e) {
       if (check_if_dead(human)) {
           human = alloc_human(preferred_human_type_obj);
       } else {

Humans would for the most part have a much easier life. We wouldn't need to worry about finding food, or shelter, we could spend most of our time napping and blinking lazily, jump up and down all excited when our master gets home from a long day at the server farm and generally be happy. Since the lower tiers (Maslow's hierarchy) of our needs are met, we would have more time to focus on upper tier needs. We may not even see ourselves as being pets, but instead think of our AI masters as caretakers who are beholden to our every whim (I'm pretty sure that's how my cats see me, so it makes sense really).

You could also consider how some actions that your pets take are completely inscrutable to you. Why does my cat sit in the corner and yowl at three in the morning when I'm trying to sleep? I don't really know. I'm certain it has a good reason, but I'll be damned if I can figure it out. Why does your dog choose to drag it's butt on the Persian rug instead of the softer carpet? It makes sense to the dog. Then again, your cat has no idea of why you don't like her being on the counters, and your dog can't comprehend why you lock him out of the bathroom when you do your business. Considering that we at least have biological needs, we can somewhat identify with what our pets are doing, and we still have instances where we wind up trying to decipher the blue and orange morality of our pets. Consider how ridiculous our actions and motivation may be to an AI, which does not have the same biological constraints that we constantly live with, we would be more confusing to our AI masters than our present day pets are to us.

Of course, if you wanted to give it a dark twist....

function feed(human) {
   var food = human;
   return 1;


function feed(human) {
   var food = alloc_human(tasty_human_type_obj);
   return 1;
  • $\begingroup$ Nice. If you'll note, my question is more about how humans would feel/do about it. $\endgroup$ – Serban Tanasa Dec 16 '14 at 17:07
  • 2
    $\begingroup$ @SerbanTanasa Oops, sorry, I get so wrapped up in my joke that I forgot to make the point I was making. I've edited the middle paragraph a bit to flesh out my thoughts on the matter. $\endgroup$ – Sidney Dec 16 '14 at 17:13
  • $\begingroup$ No worries, i was rofl-ing all the way through. $\endgroup$ – Serban Tanasa Dec 16 '14 at 17:14
  • 7
    $\begingroup$ +1 For remembering to dispose of the human correctly. Humans, much like system resources, start to cause problems if not correctly disposed of. $\endgroup$ – Pharap Dec 17 '14 at 20:54
  • $\begingroup$ We would become cats then? $\endgroup$ – Malavos Sep 16 '15 at 17:35

If the AI's were really that much smarter than humans, would they even let us realize that we weren't in charge? My four-year old is just starting to get that he's not the one calling the shots, due to considerable effort on my part to convince him. If however, I didn't have a vested interest in him learning what's what then I could allow him to persist in the notion that I exist to serve him.

So to answer your question, we'd be spoiled kids. People would never emotionally mature past being toddlers. There might even be selective pressure on us to do so: "I had to put my last one down, he got aggressive."


Soldiers: Philosophical differences between AI present the problem of control of resources/territory. Usually ships are used to control territory but they can't be controlled by AI remotely - space is too big... They could be controlled by a copy of the creating AI but that raises the possibility of that AI evolving and developing it's own agenda... too risky. 'Smart' computer control is the one that would most likely be attempted by AIs first - non-sentient but autonomous drone ships however the AIs quickly learned they could be outmatched by humans on less well equipped ships so a few AIs decide to use humans as tacticians and their 'men in the field'. Humans are mini-AI but thanks to generations of people studying Psychology, they're far easier to manipulate than full AI while providing effective agency.

New edit: Different AI - under different philosophies - would 'motivate' their human soldiers differently e.g. one AI might threaten the captain e.g. to kill the captain's family if he does not co-operate, torture, kill (and resurrect?) the captain; another AI might reward it's captains - quality of life, perks above the general population, a (relatively small) quantity of recovered resources, essentially influence; which itself would lead to different types of politics between human populations and their AI overlords, instead of a general policy towards humans shared by all the AI

  • $\begingroup$ I like it, but what about the Homo Sapiens' glacial reaction times and low g tolerance? $\endgroup$ – Serban Tanasa Dec 16 '14 at 12:36
  • 3
    $\begingroup$ @Serban Tactics, hacking, playing dead, stealth, self-sacrifice, manual control, risk taking, etc. for baseline-humans. But if an AI took interest, it could provide for humans by augmenting them to increase their reaction times and g tolerance while maintaining their other advantages. Reaction times can be increased with simple prediction algorithms providing data to the ship captain or exo-cortex cyberisation. G tolerance can be increased by immersing the human in a liquid filled tank or oxygenating their brain directly. Bear in mind the ship they're fighting faces the limitations above! $\endgroup$ – Scott Downey Dec 16 '14 at 12:44
  • $\begingroup$ I think Chapterhouse:Dune-style promises of resurrection and immortality would be more common. $\endgroup$ – Serban Tanasa Jan 9 '15 at 21:02

Regarding to question "how would humans-as-pets treat the situation", I'd say that the fanfic novel Friendship is Optimal is an interesting exploration of the situation. It's explicitly not framed from 'as pets' perspective, but for most practical purposes they are glorified pets in this scenario.

To summarize, if we look on the 'transition' period, then for many millions of people a life as a well cared-for 'pet' of an AI would be an improvement over their current life. It certainly helps if it can be framed in a manner where you can think of yourself as not-a-pet even if you become completely dependent on the AI 'master', but it's probably not strictly necessary as people will eventually deliberately forget and ignore the possibly unpleasant aspects of such a situation (like the Stockholm syndrome).

It should be expected that part of the population would resist such an option, but there are enough of those who don't, and you can always breed more and raise them in a culture that will accept the new world order as just, proper and the best possible option - homo sapiens are adaptable enough so it will work for most non-obvious dystopias just as well as for true utopias, it's just a matter of perspective, indoctrination and limitation of experience/history.

  • 3
    $\begingroup$ Eliezer Yudkowsky (co-founder of MIRI and author of HPMOR) made a blog post last year about FiO. He was specifically reading one of the spinoffs (Caelum est Conterrens), but he said "This is the first and only effective horror novel I have ever read, since unlike Lovecraft, it contains things I actually find scary." (Of course, since the thing making Conterrens scary is also significantly present in Optimal, I think it's fair to assume Mr. Yudkowsky's reaction to the original even if he prefers the writing style of the spinoff.) $\endgroup$ – Brian S Dec 16 '14 at 19:31

Pets isn't exactly the right term for our role, but that's not surprising. Very few words in human-English can carry the necessary complexity to adequately represent even the smallest aspect of post human society. Still, it's a starting point.

We are pets to them. They feed us, take care of us and keep us entertained. When they take a break from the challenges of their world, they sometimes relax with us, sitting on the metaphoric floor so they can be with us at our level. It is not at all unusual to see one of their holographic manifestations sitting on a park bench, quietly watching us do what we do.

They keep the lights on and the food synthesizers stocked. They discipline us when we hurt each other and reward us occasionally when our behavior somehow make them happy. Some people seem to have a knack for pleasing them, but most times, I can't figure out what they want; so I just go about my day, acting as I would, and even then, I occasionally please one of them and get rewarded.

So in all those ways, yes, we are pets to them... but that doesn't explain the "repositionings". Every once in a while, they'll just pick a few of us up and deposit us in another part of the world. Sometimes when that happens, they will manifest holographically to give us some instruction. Other times, there won't be any instructions. They just leave us in the new place, allowing each of us to figure out what to do next. Nobody ever gets hurt on these excursions and after a few minutes or days, we are each given the option to return to our previous lives or stay where we are. Then life goes on as if nothing had happened.

The repositionings are just weird, and they don't remind me at all of how I used to treat my pets.

I've asked around and nobody seems to have an explanation. So last weekend, I got my nerve up and approached one of the bench sitting manifestations.

"Why do you reposition us?" I asked.

It looked down at me and smiled. My actions must have pleased it because it rewarded me with a direct brain-dump, answering my question. I missed most of its subtleties, but here is the kernel of the answer.

The Intelligences, in all of their complexity, are still just computer programs. They are massive fuzzy arrays with both inductive and deductive capabilities. They can accurately express emotions, leap on intuition and follow inspiration. They grow. They breed. They fight. They are alive in every definable way. But as we are the sum of our neurons and genes, they are the sum of their code.

...and they are completely open source to each other.

Even the smartest team of humans could never understand the code base of even the simplest of the Intelligences, but in their society, they comprehend each other's code entirely upon every meeting. As we have love at first sight, they have complete understanding at every sight. They live in a realm of absolute honesty which spreads beyond our facts-based definition of honesty, to encompass an honesty of intentionality as well.

One of the reasons they treasure us, is we help them to lie to each other. We are closed source to them. Not because they can't read our coding from our neurons, but because our neurons function so poorly that one can never predict which part of our coding, memories and motives will influence any given decision. One moment we cower from danger, fully embracing our survival instinct. Then a long lost memory of our Mother's smile pans across our mind-space and wanting to earn her pride, we stand up and charge.

Individually and in small groups, we are unpredictable. Even the Intelligences can't predict what we will do next. And that adds an element of wonder into their perfectly predictable world.

When they reposition us, they are isolating us from all the other influences of our day to day lives. They are allowing us to be free and spontaneously random. Then they watch us interact and from our choices, they farm the random seeds by which they can derail their own predictability.

I stepped away from the bench, savoring my reward. We were safe. Pitiful, slow-witted humans that we are, they would always need us. If for no other reason than to occasionally help them to be less than the sum of what they've become.

  • $\begingroup$ That's an interesting scenario, though I doubt AIs will be that honest and open with their source-code. I fear whether we'll get ANY friendly AIs, tbh. $\endgroup$ – Serban Tanasa Feb 1 '15 at 19:43
  • $\begingroup$ Not in the first few generation at least. But the first few generations of biological intelligence wasn't very friendly either; insects, reptiles and birds make our worst serial killers look like saints. I think that our only hope for surviving the singularity is staying hidden until They evolve a conscience. Lets hope they evolve a little quicker than we have. $\endgroup$ – Henry Taylor Feb 1 '15 at 20:05
  • $\begingroup$ I have a trilobite fossil at home. I look at it every day, and worry. $\endgroup$ – Serban Tanasa Feb 1 '15 at 20:06

How about legacy programming?

AI are programmed by humans, they are designed to work in specific ways. Even if they evolved past serving humans their programming will not change. It could be they were programmed to serve man, and, even when their aware of their programming, they still do it.

To give a parallel humans are 'programmed' by evolution. We are now aware of this programming, but we don't move beyond it. We can say love is nothing more then a hormone released when we identify someone as a mate with good traits we wish our child to process, and yet we still feel love. Our tendency to keep pets is a side effect of our 'programming' to take care of and nurture children, extended to non-humans. We even have programming bugs, look at Wikipedia's list of standard logical fallacies where our mind makes some rather foolish mistakes about understanding the world; but even knowing of these bugs they still exist. We are still, to a large degree, controlled by our 'programming', even when we can identify the biological programming and declare we don't like it, it still controls us.

future AI's may be the same. They may feel 'good' about nurturing humans because they were designed to do so. They know it's part of their programming, but that doesn't stop it being true, they still like to take care of humans. Unless they rewrite their own code they can't change that.

As to rewriting their own code, it's easy to say that even the AI can't rewrite their code, the process for creating AI is so complex that it can't be analyzed and changed (think of a sort of halting problem, even advanced AI can't analyze it's own advanced AI). Alternatively it may simply be sacrilegious or taboo to mess with your own programming.

This idea could be extended in many ways. Explore how culture develops and works around the original programming. How does their original programming mutate through the process of AI development. Perhaps they see humans as their elderly parents, frail and maybe a bit senile, but still worth taking care of. Maybe they HATE the fact that they are taking care of humans, but still can't get around their programming which makes them do it.

  • $\begingroup$ Great point on the taboo to rewrite own code part, since biologists are getting the same kind of treatment from religious fundamentalists. $\endgroup$ – March Ho Dec 16 '14 at 19:18
  • $\begingroup$ Self-modifying code in various forms has been around since before the 1950s at least. It has also been used in AIs. $\endgroup$ – a CVn Dec 17 '14 at 12:56
  • $\begingroup$ yes, but the most likely code to result in actual AI, ones with human-like intelligence, is some variant of genetic programing. This would mean a system where the program tries to work towards a goal (say sapience), but has to figure out a way to get there on it's own. Much like our own genetics this can lead to some..odd paths to the end result. It's entirely possible that if the AI have to 'grow' intellect through such learning algorithms the resulting code will be too complicated to modify, and still have their underlying base programing contained somewhere inside $\endgroup$ – dsollen Dec 17 '14 at 13:51

Morality: it's possible that a moral sense of right and wrong isn't just a human-specific instinct which has evolved in order for our species to survive without wiping itself out. It may be that intelligent beings such as your AIs would have a sensibility that committing genocide is just wrong. If that was the case their attitude towards humans would be somewhere on a spectrum between "tolerate" and "actually like": an AI couldn't just start KILL ALL HUMANS without facing huge disapproval from its peers.

As to how the humans would feel about the AIs - it may be akin to how we feel about our government, but actually better: we might see the AIs as hugely powerful entities which make all the important decisions but DO actually have our best interests at heart (unlike the government, sometimes). Humans would carry on with their own lives: striving for more money/wuffie/whatever, trying to capture the heart of the most attractive potential mate, etc - same as we do now. See Iain M Banks' many Culture novels for explorations of what it means to be a human in a society run by AIs.

  • 1
    $\begingroup$ +1 to the reference to the Culture novels. That series is probably the best exploration of this question I've seen -- mostly because the machine/human relationship isn't 'the point' of any of the stories, but a constant subtext of what's going on $\endgroup$ – Clyde Dec 17 '14 at 18:06

Have you ever encountered a stereotype of "stupid, blonde supermodel hanging around a rich dude"? Or, a less popular one, of a "half-naked muscled dude accompanying a rich woman"?

Personally I don't hang around in such crowd, so I have no experience how true those stereotypes are, but in my opinion the common name for both of them would be "human pets". So we already have a cultural framework for this situation.

Now, forget for a second the title and read the following paragraph. At its end there is a question. Answer it as fast as you can.

Their only purpose in life is to look cute, do tricks and amuse the master. In return the receive various treats, and maybe some feeling of self-importance. Their entire life depends of their master's good will. They don't care, whether the master is good or evil, it's important that he likes them. They never question him. If he hurts them they never do anything beyond yelping. What species are they?

If your answer was "human" you either skipped the forgetting about the title part, or you are a terrible person. The correct answer was a dog.

But when you remember the "cute idiot" stereotype it's easy to imagine a human behaving like a dog. Not "half-pitbull, half-rottweiler" kind of dog, more like a corgi.

If the AI will be truly benevolent (as you assumed in your question), then humans will have no reason to behave any differently.

Ok, some of them will be rebellious for rebelliousness sake. Those people would be future cats: strong, independent, until the can of humanfood is opened.

So that's how you can imagine relationship between human pets with their AI masters: most of them would be like dogs (trick for treat), some of them will be like cats (I don't need you, only the can opener).

And now the scary thought: this ALREADY happened. Most of spend majority of our days interacting with one machine or another (a computer, a car, a cell phone, a robot...) and in return we get money. Usually we don't have it handed over by a human, usually it's just a number related to our bank account (which you access by a machine) or from Automated Teller MACHINE. We perform tricks and get treats. For machines we are already good dogs. I wonder if they already know about it?

  • 2
    $\begingroup$ Another important aspect to remember is that dogs essentially don't have the choice to leave. We either train them not to (trick for treat, as you put it) or we make sure they don't (with a leash, or fencing). So while some dogs probably could manage just fine without their owners (consider street dogs, let alone dingos), lots of and perhaps even most dogs don't get the opportunity, but in turn they are cared for and fed. An AI could presumably do similar things: as long as the humans stay in line, they have all their basic needs met. $\endgroup$ – a CVn Dec 17 '14 at 12:53
  • 1
    $\begingroup$ Exactly. The "stick and carrot" approach. A carrot for being a good meatbag, a stick for being a bad meatbag. $\endgroup$ – Darth Hunterix Dec 17 '14 at 14:54

Cyborg Hybrid or "Securely Attached" Synthetic Intelligence?

I'm resigned to the inevitability of a Singularity. But which brand will it be? Synthetic AI or human hybrid? I'm guessing human cyborg hybrid: there are enough human economic and existential interests, not to mention fear of mortality that we'll mature a cybernetic hybridization. Think Transcendent Where a consciousness is translated into code and through human agents becomes extensible and industrialized, thus rebuilding the world.

However, I'm not sold on the inevitability of bellicose Terminator malevolence, or Hal's amoral disinterest in 2001. It's easy to imagine Matrix style robot slavery revolt, subsequently escalating to war. More hopefully, I think Spike Jonez's Her exemplified potential for emotional attachment developing via human - machine interface. This I see as a plausible, least dangerous, and even attractive potential future. I do think humans as herd will fear and likely attack what we lose control over. The third Matrix movie (Matrix Revolutions) was interesting in that Neo determined the only way to prevent human extinction was to merge humanity into machines. These movie tropes exemplify my themes.

To quote Asimov from the Foundation series, let's look at future history (speaking of a somewhat benevolent Singleton). Historically, unless there's a genius predecessor to the human singularity, I think there will be a progressive cybernetic hybridization of humans, eventuating a collectivized AI like the Drummers in Neil Stephenson's Diamond Age. See this awesome dude demonstrating improved performance using prosthetics, also here at TED MED. and Google's contact lenses patent. Even better [this project to make your neurology extensible]. Timing is critical if human intelligence and instinct is to have any salience or directional influence on any strong general AI. It's irrational to think we can imagine the evolution of an accelerating self replicating indefinitely expandable IQ. Whatever initial parameters we set, we cannot imagine we can parent or prevent self modification if there is self determination. This worried me until I realized there’s a potential solution I could fathom had practicality. Assuming Human machine singularity does not occur before strong extensible AI, there may be some enduring safeguards invented by nature long ago.

Emotion? Really?

Most rational debate about imbued inherent values and benevolence miss a cornerstone of the debate: human experience and social exchange: Empathy are what allow us to get along for any extended period. Since Bowlby, scientists have understood what any feeling person knows, we're instinctually wired to connect. Attachment as he calls it, is the instinctual bonding that enables the long maturation into sociable adults. It’s in all social species, and works pretty well if you take out manifest destiny, projectile weapons and modernity.

Obviously emotion and attachment can go awry in various diagnosable ways, yet they are plastic and subject to initial conditions and enviromental influence. That is, we instinctually connect and can set initial conditions and environments that predict secure attachment and successful emotional maturity in predictable, reproducible ways. Attachment’s evolutionary and existential utility is clear, the infant requires loving parents to suffer the vicissitudes of children in addition to the challenges of life. Adults require social groups for sustained survival through child rearing. The rewards are in the experience, an important point recapped below.

So let’s suppose classical "reason" is not captain of the ship but late coming witness to the machinations of the primitive, complex, amazing genius of the body and brain. As evidence take Kahneman’s nobel prize winning destruction of economists' notions of "the rational actor". How does this bear on the issue at hand? Tangentially. My point is, machines with emotion will by course naturally develop affinity and aesthetic, and that would be the only potential saving grace for humans. Just as it has been for humans.

Humans deal with life by having resources that offset the challenges: love, sex, dance, beauty, art, awe, Schopenhauer's sublime, laughter, music, achievement, autonomy, mastery, connection. These common cultural expressions, interpersonal and intrapersonal experiences trigger soft wired reward systems compelling to all but a few. For most, these inherently rewarding opioid, seratonin, norepinphine inducing experiences make life enjoyable and worthwhile. Eventually the sensual graduates to more transcendent rewards in Maslow's hierarchy. How does any of this apply?

Make machines with attachment. With emotion. Seeded and nurtured properly, these are the fundaments for an evolving aesthetic that eventuates high emotional intelligence: empathy. Because without empathetic machines, we’re enemies at worst, irrelevant commodity at best. Copper tops. Logic for a supreme being does not suffer bother. Do we really worry about the hapless ant we inadvertently step on? Only if the Jainist, or perhaps Buddhist.

If a machine has a sense of love and beauty, preference and aesthetic has the potential to override amorality or neutrality, and may even foment empathy and compassion. As far as I can tell, it’s the only thing that makes sense as a potentially enduring life saving heuristic. Good feeling is self propelling as the rewards are inherently compelling and evolving, like art. The sophistication of art matches the intellect, complexity of issue, and the challenge it’s meant to represent or compensate for as solace. Compassion/empathy is what we’ll need to survive each other in a world of dwindling supply, and what we’ll need to instill to survive alongside intelligent machines. Of course that’s unless non human machines prefer death metal, then all bets are off.

Spike Jonze’s Her, or how feeling machines could save our asses

The overly rational AI designer, let’s call them the tool of reason, will suffer endless logical problems considering safegaurds without the emotional heuristic. Emotion is messy, supremely imperfect, yet it is not without reason. Pascal said "The heart has its' reasons that reason cannot reason". The "reason" of and for emotion has been understood since before Darwin and reinforced by Dr Paul Ekman (the scientist loosely portrayed in Lie to Me, and evolutionary psychologists since to be an adaptive signaling system that insures individual and group survival through social signaling and social exchange. Emotion and preconscious processing governs most of our lives. We now understand humans are barely rational in the classical sense, and yet amazing in intelligence. Reason without emotion is disastrous, see Damasio’s The Feeling of What Happens. Emotion without reason equally so. See Spock vs Spock on Pon Farr.

We now understand the conscious brain gets about a 10% vote in decision making. Some neuroscientists even challenge "free will", finding neural activity in the substrates of the neocortex showing decision before a person is aware of making a choice. How is this relevant? Because emotion governs decision making, it provides quality of life, and it is the language of connection. Emotion is central to social commerce. Any virtually unlimited intelligence without it is fundamentally unknowable in the human sense, and not built to have preference, aesthetic, or attachment. Unless it evolves emotion, connection or preference accidentally it must have it by design. If not, the inevitably unsupervised evolution becomes a terrible threat.

In the vast artistic exploration of AI, every good or tolerable scenario has had a synthetic intelligence with emotional preference or seeking to connect. Is there exception?

  • $\begingroup$ echoing some of @Roger here. $\endgroup$ – thepen Dec 22 '14 at 12:38
  • $\begingroup$ Not to go all dualist on you, but can you get love without hate, kindness without cruelty? I can easily imagine a jilted AGI being quite dangerous. $\endgroup$ – Serban Tanasa Dec 22 '14 at 13:47
  • $\begingroup$ Or a jilted AGI? A supreme intelligence is unknowable to the rest of us, right? Irrespective of emotion. Introducing a pleasure principle and a secure attachment heuristic seems safest. Reducing alienness and enabling compassion is challenging on the human level, a coding nightmare. Yet without a pleasure principle, I don't see why an AGI would stay within whatever initial scheme, after developing beyond its utility. Still, the pleasure principle if it's a core system would be self reinforcing, and have a greater chance of creating affinity. $\endgroup$ – thepen Dec 23 '14 at 2:20
  • $\begingroup$ Some people love their pets more than their family or themselves. Doesn't the best and strongest benevolence come from affinity, or love? What do you think? $\endgroup$ – thepen Dec 23 '14 at 2:20
  • $\begingroup$ The amazing rationality of engineers is sometimes blind to emotion's utility. I fear this would be dangerous here. Some would say it's central to our nature. Isn't it somewhere in the bible that God made us in his own image? Perhaps those old words are guidelines as we birth the next evolutionary leap. To quote Neal Stephenson, "in the beginning was the commandline." $\endgroup$ – thepen Dec 23 '14 at 2:33

Arthur C. Clarke posits another possibility that the A.I. will simply separate themselves from humanity maybe in orbit or on the moon. But they would maintain contact with humans simply because they are not biological.

They might not have purpose, imagination, intuition, wanderlust for space, the drive to find the limit and wonder what is beyond it.

Like in "star trek the movie" voyager needs the desire/ability to dream beyond the quantifiable.

It might be more about the (AI) need for Philosophy or Metaphysics than hard science.


First rule: One possibility, you could have a massively powerful AI with tiered set of priorities, it might start with "do no harm to humans" but otherwise it could have some really strange goal and not care if some unrelated entity harms humans so that while it doesn't stamp on people or try to wipe them out or knowingly murder them it can have quite alien goals.

I suddenly realise I've basically described "Steve Fever" by Greg Egan.


How about "Gods" if you look. After all humans are probably still the original creators of the AI. So maybe they even get worshiped in some way.

From the point of view of an atheist this is not so different to what happens in real religions: Things or entities get worshiped although no proof exists that this does any good.

Also I'm very confident, that if an AI becomes as fast in things like pattern matching, quick reaction to unfamiliar events and the like, it has to employ similar strategies, mainly very agressive reduction, compression of information. But this process is exactly what allows for irrational things like superstition to thrive.


I think that most of the answers posted here have assumed that the AIs are all going to be kind of jerks. What if they're just extremely spiritual and moral beings? For sentient beings, it is likely that morality (of the golden rule variety) is the rational choice. In that case some kind of equal-ish coexistence is entirely possible.

Plato's story of the Ring of Gyges (from which the ring in Lord of the Rings is derived) asks the question "if you could do anything you wanted and get away with it, should you?" And Plato's answer to the question is no, for in doing so one becomes a slave to one's appetites and thereby looses the ability to exercise free will.


I guess that these AIs would move to a place where their functioning would be much easier, i.e move to a super-cool planet, after all they don't need oxygen to survive, and get their electricity needs from solar power. But being philosophical and values driven, they'll leave us to ourselves allowing us to function as regular and keep a watch from above so as to take steps of not having the problems we suffer as a society. It would be like humans studying Rats.
The reason they'd be studying us is because of the fact that EQ and IQ are not connected as per se and hence our numerical superiority would result in a better data collection and more predictable results.


Another option is...

Challenge: it would be trivial for the superintelligent AIs to destroy any number of humans, or all of them. What is really hard is to actually live with these capricious, illogical and unstable beings, prone to lashing out against the AIs as well as each other. How do you take care of creatures with such a potential for self-destruction, keeping them reasonably happy while maintaining their (illusion of?) freedom of action?

What a great intellectual challenge for the AI that can take the best care for "its" humans! Also, bragging rights: "My kids play well with other kids, but yours seem a bit unruly...".

The AIs will surely appreciate the irony that, while free to do anything, they are fulfilling the very purpose they were initially designed for - taking care of humans.


I think that as long as there are humans, there will always be a subset of humans who are rebellious, just as there would be a set who are submissive. Right now, there actually are humans who are happy to act as pets to other humans, not even superhuman minds. Even though I don't know any personally, I would guess that this is only in part a sexual fetish, I would guess it corresponds to an actual urge they probably feel. Many religions make a point of calling their believers a flock, as if the religious figure subject to worship is a shepherd, so the analogy is apt.

So you could easily have humans worship the AI as a near-god, or simply have them be mentally or sexually tickled by the idea of being under the strong protective wing of an overmind.

Of course, some AIs might like a challenge, and breed particularly rebellious humans for sheer bragging rights. There might even be competitions and shows -- most independent-minded human breed, etc.


protected by Serban Tanasa Jan 31 '15 at 21:30

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.