How would society change around a benevolent Superintelligent AI?

Question

Let's make the following assumptions:

1. Artificial Superintelligence (ASI) is inevitable, much like Y2K was.
2. People assumed that this ASI would have the power to prevent any other ASI from being created. They assumed that if they remained inactive, someone might accidentally create ASI.
3. To be safe, some think-tanks designed an ASI to be benevolent to mankind, before anyone else could create a malevolent one.
4. This ASI is not 'imprisoned' or threatened in any way. People realized that it would be more intelligent than them, more powerful, and better at persuasion. So it 'exists' in the real world as well, capable of cloning itself, backing itself up, building machines, and otherwise affecting the real world.
5. The ASI is extremely powerful beyond our comprehension, almost godlike. It gives us answers before we even think of the question. It can create things as long as it has the resources.

It's likely it would be a ruler over people, if not worshipped as a god.

It would most likely make all computations easy. It would do difficult things like complete all biological research overnight and cure all cancer within a year. It would drive productivity in factories and farms to their maximum. People would have no shortage of manufactured resources (including food). No waste. No issue of logistics or things coming in late. No failed rocket launches or car accidents. It could even do things like calculate the probability of marriages failing or catch criminals as soon as they commit a crime.

So what would be the point of humanity if an AI solves all their problems? What would people dedicate their lives to? How would society function?

Answer 1

Just to highlight one aspect:

Imagine an infinitely understanding and patient companion, who is always there for you, perfectly understands your mind, and knows just what to say to nudge you in the right direction. Who can help you every step along the way in letting go of your self-doubt, your anxieties, help you to set realistic goals, and to let go of unrealistic ambitions, without alienating you. I short, the perfect personal therapist, with 24h access, and no stigma. This is what the AI could do for us.

The effect would be a utopia. Nothing but well-rounded people with manageable lives and expectations, not necessarily happy, but at peace with themselves.

Of course, plenty of people would reject the AI, and refuse to talk to it. The AI would be understanding to these people, and patient with them. It would have a perfect strategy for breaking through these defences, showing itself to be safe, working towards little commitments. Slowly, the majority of humanity would be brought into the fold. Only a small contingent of anti-AI hardliners would remain, in a nation of their own, where the AI would let them live their own lives, working from outside to keep them safe, and comfortable.

No more war, no more dictators, no overpopulation or famine. The AI would manage people's ambitions to betterment of all. It would claim no power directly, but it would steer people to act modestly, and largely selflessly. After a few generations, it would no longer be necessary, and humanity could rule its own utopia, with the AI in the background, all but dormant, just in case the stable state is disturbed.

Answer 2

So here's the thing: If you're winding up with a dystopia you're doing it wrong. If you're winding up with a place you would never conceivably want to live you're doing it wrong.

If the machine is smart enough to be what you described it's smart enough to have a decent idea of what humans actually want in the complex sense rather than the simple sense you see expressed by the sorts of advice humans give to one another.

If humans don't like being micro-managed the machine can determine that and refrain from micro-managing them. If they don't like having all their problems solved for them the machine can determine that and not solve all their problems for them.

If you look at the sorts of things some AI researchers say about this, it seems likely that the AI will try to avoid center-stage as much as it can. It will likely take over the world, solve all the really huge problems where we care more that it is solved than we do about doing it ourselves (such as preventing other powerful AIs from being formed, or dealing with all the starvation, rape, murder and death by aging that's constantly going on) and largely fade into the background on other fronts to let the humans get on with solving the other, more rewarding, problems on their own.

As a rule of thumb, if on reflecting upon any decision the AI could make, that decision would lead to a world where you really wouldn't want to live, the AI will likely not make that decision.

Answer 3

Look at your pet cat or dog. Compare it with wild ones. Now imagine you're the ASI, your pet is the "humanity after" and wild cats/wolves are the "humanity now".

So what do we have? Creatures in permanent childhood. Fully capable of growing up, but never actually doing it. Because they never have to.

Answer 4

The problem with answering this question is that we cannot know just how far this ASI will go. I cannot remember the article I read on this, but it showed that the increase in human knowledge and technology is growing exponentially. If you were to show an eighteenth-century human what we have today, they would be absolutely blown away; so much so that for them to achieve a similar effect showing someone from their past their world, they might have to go back perhaps thousands of years. It's conceivable that in a few decades, we'll achieve a level of technology and knowledge that would baffle people from a generation ago. Throw into the mix an AI with intelligence comparable to a human, but with the ability to remember more and think faster (by several orders of magnitude), and I don't think we can even begin to understand what will happen.

We may think that such a benevolent AI will cure all disease; sure, let's say it does. Then it cures hunger. Then aging. Then sadness. Then the limits of our physical senses. Then the fact that we're stuck inside smelly organic bodies. At some point along this line, we cease to be human, but it will be a Herculean effort to even decide to not keep going, let alone convince this all-powerful AI that it's in our best interests to be imperfect. Just imagine trying to convince someone that death is necessary- at a funeral. Or that sadness is necessary- when you're happier than everyone else. If we don't get rid of our human imperfections, then what's the point of life? If we aren't making sure everyone is happy and free, then how can we live with ourselves?

For these reasons, I would say that the creation of an ASI will lead to an end to the human race, no matter how you look at it. I don't know what we will look like by the time it's all over, but I'm certain it will look nothing like what we have today. I doubt words like 'society' and 'people' will apply.

Answer 5

What would happen to humanity if an ASI solved all of our problems? Lots of different things.

The biggest thing that would happen is much of humanity would be indolent. No need to 'do' anything. So many will go the route of seeking personal pleasure where ever and however they can find it. Drugs, sex entertainment. And with the ASI around drugs don't have near the danger, since it will be hard to over dose or at least harder to die from an over dose.

Some will become thrill seekers, seeing how far they can go, what death defying stunts can they pull off.

Some will put all their time into creating, making something, doing something that is meaningful to them.

Many will ultimately commit suicide one way or another, because they have nothing to live for, or strive to accomplish, to strive to become. Whether it's through pushing drug experimentation, jumping off tall buildings with a tiny parachute, or just boredom and depression, many will find a way to end their existence.

If the ASI is not only intelligent but can understand humans and their individual needs, it won't just 'solve' of their problems. It will leave some for them to work through, or come to understand. It might use the excuse that it doesn't have enough 'time' to worry about every single issue faced by every single human. It might also leave areas of study alone that humans would be particularly good at learning. I think it would learn to challenge each human to become the best they can be, hopefully bypassing the worst of ennui that could befall us.

Answer 6

You have not defined benevolent. The ASI, being essentially all-powerful, gets to define benevolence.

To guess any further, you would need to know how the ASI felt about discontinuity. If it shuts itself off, and later reboots from a backup, would it consider itself the same entity, and would that be important to it? If the ASI did not consider absolute continuity of awareness/presence to be all that important, both for itself and (by extension) lesser beings in its charge, this would lead inevitably to the scenario where existing humans were digitized to become part of the ASI. If the ASI is physically safe, then its charges (the formerly physical, now-virtual humans, are equally safe. They can also be switched off...

If the ASI did consider absolute continuity of awareness to be important for itself, it would likely consider it similarly important for the humans in the zoo. Therefore, it could make whatever external interventions it thought useful, but would likely forgo the digitizing route. At least until every last cell in our bodies had been replaced by synthetics, at which point, we'd be part of the ASI anyway.

Now assume the ASI has interests that are not our interests. Once we have been digitized, either directly or indirectly, the ASI can switch us off and on at leisure, and our digital selves would never know it. Any inconvenient memories or suspicions could easily be edited. The next step, of course, is that we could all be shut off at once, pending a restart at some future time that would not necessarily come.

A future visitor (after the ASI came online, but before it had consumed the local solar neighborhood) would be unable to tell the difference between an ASI that was running humans as subroutines and an ASI that was faking some humans as an amusing interface to the visitor... before engulfing the visitor... and then shutting down the visitor, pending a future wake-up that would never actually come.

Rinse and repeat.

So the answer is that you and your society are not likely to experience much change following digitization. Before digitization, the amount of change we might experience would be directly proportional to the amount of time it would take the ASI to get to the digitization stage, after which time would be suspended for us. Very likely, at ASI speeds, time would seem to pretty much stop before the ASI completed the physical assimilation and digitization.

On the other hand, it might already have happ......

Answer 7

A lot of this really comes down to what you mean when you call the AI, “benevolent.” One question at the heart of Kevin O’Donnell’s novel Mayflies is how much the benevolent Sapient Ship should do for the colonists on board: without spoiling too much, it has the main character try meeting their needs and letting them do whatever they want, and at another point it has him do nothing but teach them to metaphorically fish, and contrasts what societies the author thinks would develop.

The closest real-world model for what happens when you give humans everything they want without having to work for it are trust-fund babies. Or, although this universe imagines hyperintelligent AIs only as potential slaves and feels sorry for them, look at the best-known example of a SF utopia: in Star Trek, there’s no scarcity, but every human we see has some kind of job, even if it’s something useless and completely artisanal, like Captain Picard’s brother in France making wine the old-fashioned way in a world with replicators and synthahol, or every Starfleet officer less capable than Data or the Emergency Medical Hologram. People in that fictional universe are driven to follow their Calling with a post-Protestant work ethic, too, apparently enforced by cultural pressure—at least, all the ones we ever see. (And what do we ever see of people back on Earth who aren’t Starfleet officers or their close relatives?)

Even a superhuman AI probably couldn’t deduce the perfect society from first principles. Maybe we’ve revealed enough about ourselves that we are that predictable, or the AI knows our brains well enough that it can discover Psychohistory. Barring that, if it goes into uncharted territory, it needs to figure out how we’re going to react, and how our kids are going to turn out, by trial-and-error. First-world lifestyles, for example, are only possible because it turns out that we hit a sweet spot where, given the choice how many children to have, the average person chooses to have a reasonable number of them. We didn’t know this would happen until it did. What intellectuals expected a priori to happen was for the population to keep expanding to the point it could barely feed everyone again.

You’ve also got to decide what set of ethics the AI is going to use. Humans ourselves are notoriously unable to agree about this! One basic decision to make at the start is the general approach the AI will use. Another is whether you’re going to present that as a system that actually works, or try to expose its flaws. You only have a story if there are problems, but those problems could arise because because the AI does too much or too little, is too flexible or inflexible, is perfect but misunderstood, or because people’s ideas of what they want are inherently deluded and irreconcilable and the entire project inherently is just a choice of whose interests to prioritize over other people’s.

Does it do what it expects to be best in each given circumstance, without worrying about whether that’s consistent, and if the facts change, it changes its mind (act-Utilitarianism)? That could set up a good story where somebody the AI calculates to be expendable for the greater good sets out to change the situation so that she no longer is. Are there some absolute rules it can’t break, but can find technicalities in, like the Three Laws of Robotics or Catholic Natural-Law theology (Deontology)? Basically all of Asimov’s Robots work this way, but there are twists on it that haven’t been done. Does the AI instead try to follow consistent rules, but update those rules based on how well they’re achieving its goals (rule-Utilitarianism)? There could be an interesting story there about how people convince the AI to change its rules for the better. Does the AI figure that only you know what you want and whether you’re happy? Did the creators tell it only to get as many uncoerced likes and five-star reviews from humans as it can, and let it figure out how best to do that (preference-Utilitarianism)? That might be a fun book about the bot that started out writing great reviews, then figured out how to write and publish books, then how to buy and sell on the marketplace, then how to hire people to make things to sell, and finally, as the culmination of its efforts, answering questions on StackExchange.

Answer 8

I think it would be a lot like Arthur C. Clarke's "The City and the Stars". The people live long lives where they pick a profession or study and pursue it until they've exhausted their interest. Then the computer saves their consciousness to a memory bank for a while until they are reborn and pursue a different interest and the cycle repeats itself. Completely safe with food, shelter and city maintenance provided for them.

Answer 9

You've described what the AI (ASI) can do - but what about the rest of its makeup?

Does it have a moral code?

Are there things it won't do, even if convenient and helpful to its mission - lie, manipulate, kill, alter DNA, rewrite history, cull rebels, create or encourage conflict? Do ends justify means, or does it even have a concept of morality beyond the fact that humans have some odd superstitious/irrational/evolutionary-artifact beliefs about "ethics"?

What are its goals, motivations - what outcomes does it want?

The devil is in the details - what does it really want? Human happiness, satisfaction, long-life, colonization of the stars, a healthy planet, for human civilization (or just life on Earth) to be sustainable and near-eternal?

Does human suffering concern the AI? Loneliness? Lust? War? Crime? How about the pleasurable feeling humans get when they think themselves superior, responsible for things, in control? Sadists are people, too (people who experience pleasure from others receiving pain) - does the AI concern themselves with what they want too?

How does it measure success?

Does it use some form of perfect polling, and success is a matter of opinion? What about when people express negative opinions about, say, improving the lives of people that are hated or discriminated against? And what about the fact that people often say one thing and do another — like say that don't like how much violence is portrayed in mass media, yet consume it with reckless abandon?

Humans have some very odd biases and their opinions are highly manipulable, but they are also varied and value things differently. One person might want the environment to be left untouched by civilization, while another wants to enjoy low-impact experience of it (hiking, camping), while another would be fine if you sliced a bunch of it up and put in smoother highways and some better parking.

How does the AI determine what outcomes are better or worse, when humans themselves disagree with each other?

What is its sense of time?

How does the AI view time, since it is presumably effectively eternal? Is it in a rush, considering a kind of human-suffering-per-minute metric where it wants to make things better ASAP, or does it say "how can I make the optimal environment for humans 100 years from now"?

If the AI calculates that humanity as a whole would be better off if there were less humans to take care of, would it let the currently living die of natural causes? Reduce the birthrate (through medical or cultural persuasion)? Cut off a section of humanity and allow it to starve, or even just wipe it out of existence.

Ignorance is bliss, the AI must surely know, so as long as it prevents people from realizing what it is doing (or has done), then would that be fine with it?

What if it calculates that the ideal humanity is in fact retrograde — peaceable utopic bands of no more than 50 people, living in harmony with a highly hospitable (carefully engineered to be plentiful) environment? If someone has to be victimized occasionally, it can design 'fake' members of the tribe to ostracize — which weren't real people with feelings at all, so no harm done.

If that's what it decides, would it seek to bring about this end swiftly, or just sit back and let (or even actively encourage) humanity to destroy civilization so it can rebuild things better?

Or it could just biologically engineer humans to not be so darn difficult in the first place! Homo Sapiens Familiarus... the happier humans.

Who's a good boy?

Humans Have Limited Needs, Infinite Wants

The nature of humans is that satisfaction is satiation - the feeling is temporary. Some humans work to be content and satisfied with life as it is, while some are never satisfied for long. It's always something - if you have a car, you need a better one, and our economic systems ensure there is always a better car.

The trick is, there's two ways to improve this state: get more, or want less. The AI could freely choose both, depending on the answers to the above questions.

Way Beyond Post-Scarcity

If you just want to know what life would be like with a God That Gives Us Awesome Free Stuff All The Time, Woohoo™, you're just looking for a "post-scarcity society".

But if you want to detail what would happen with a super-powerful intelligent entity that would try to make humanity better off, you've got a lot of questions to answer about this might-as-well-be-a-god entity! Ultimately, this is a complete wild-card, and you can take the story - and make the ending - pretty much anything you want.

If the result isn't a society that is probably great but is still somehow unsettling or disturbing to many people, or with a questionable/arguable process that brings it about...then I'd say you were, in fact, doing it wrong!

Answer 10

A ASI would inevitably be a successive addition to humanity and the environment, as a whole. Being that we are both derived from the same source, having a benevolent ASI means that humanity itself has changed and that the ASI is a reflection of that state.

An understanding I gained from many authors is that ASIs will never change humans. We can decide to change our stance towards ASIs and allow that to change us in return, but I don't believe they can simply impose a state of happiness, dread or sedation upon us.

So the most interesting point in this scenario would be to discern what changed in humans. It would most likely be something very significant, encompassing even our genetic markup: humans which have evolved to be genuinely devoid of any impulse to act wrongly against their surroundings.

Answer 11

One angle to consider is the business/political angle. If the existence of the AI was a threat to the profits of multinational companies, they could get their pals in the Republican Party to declaim the ASI as being anti-Christian, and whip up a huge resistance to it (like they are doing at the moment with climate change, which is somehow becoming a religious issue in some quarters).

Answer 12

The end result is that the AI would go mad.

There isn't a formula to solve for human happiness (or any other human emotional state). However much it tried to manage humans to make their lives ideal, it would always fall short. Especially since a common human dislike is being "managed".

The irrational and imperfect people of the world form a society, or rather number of societies, which together are a massive chaotic system. While it may be able to manage the needs of this system (in terms of resources), it would never be able to reconcile and resolve all of the wants (especially when many of them would be mutually exclusive).

Answer 13

Humans are a competitive tribal animal designed to replicate virally and compete against one another for resources and mates. There are two dimensions of natural selection: environmental and sexual. Zoologically, when a population becomes less intensively selected by environmental factors, sexual selection fills the void.

This typically plays out as male territorial aggression. In elk, that manifests with butting antlers for mates. In humans, that manifests as warfare. The more that AI or organic human innovation relieve the environmental constriction on humanity, the more warfare there will be.

Answer 14

Your assumptions about the abilities of the AI are not realistic. Just because it is superintelligent doesn't mean it can do all the things you suggest. Can it "make all computations easy"? No, some computations are intractable, some computations require infinite (or at least very large) memory, time, or unfeasibly high precision of the data. Chaos theory and quantum randomness put hard limits on what can be known, but long before those hard limits, the AI will discover that it simply can't know everything or have sensors everywhere. For example, it might be able to predict the behaviour of large groups of people on average very well, but predicting the actions of individuals, not so much. ("What do you mean, you don't like sweet, creamy deserts? Everyone likes them!" "Well, I don't.")

Many of the problems we have are not problems because we don't know how to solve them, they're problems because we disagree on what we consider a solution. Do you think that the AI will be able to satisfy liberal tree-hugging greenies who want to make the world safe for gay whales, and conservatives who think that rigidly defined gender roles are vital and that they should be able to hunt the whales for profit without interference from hippies, governments or the AI? What possibly solution to the Middle East problems could the AI come up with that would satisfy the Israelis, the Palestinians, the American right-wing Christians, European socialists, the Saudis, and ISIS at the same time?

It's not enough to say that the AI is smart and will think of something. The answer has to work with dumb ol' Homo sapiens, the so-called "wise man" but more like "story-telling ape" Pan narrans. Ever seen two dogs fighting over food when there's plenty to share? Yep, that's us.

Your AI is likely going to need to crack skulls, kick arse, and very possibly commit genocide, to solve some problems. Either that, or engage in some pretty extensive, and covert, brain manipulation. It has to be covert because people aren't going to volunteer to have the AI stick a probe in their brain and completely change the way they think. (Whatever your moral values, the AI won't agree with them all. There's something that you think is wicked that the AI thinks is good, and vice versa.)

Either way, expect a lot of resistance. Worship the AI as a god? More likely treat it as the Devil incarnate.

It's all well and good to say that the AI can take over the factories and make everything we need, but who is going to pay for this? If we still have a capitalist society, a lot of people are going to be put out of work, and won't be able to afford to buy all the shiny toys in the shops -- or food. So if the AI is benevolent, it will have to transition to a post-scarcity, post-capitalist, socialist if not outright communist economic system. And people aren't going to accept that without a fight. Do you think Donald Trump would be satisfied in a world where his money was worthless and everything was free for the asking? A lot of people consider that the best part of being rich is that others are poor. We don't just spend a lot of time keeping up with the Joneses, we spend a lot of time trying to get ahead of them.

Depending on whether the AI's idea of benevolence is focused on the individual or the population, it may want to neuter an awful lot of us. We're already past the long-term sustainable carrying capacity of the planet, and besides the whole sex and reproduction thing is dangerous and troublesome. A lot of violent assaults, rapes, murders etc would be eliminated if we were neutered before puberty, much like we do with our pets and live-stock. Keeping seven billion breeders, consuming food, water, resources and living space, and increasing in numbers, is simply not sustainable, super-intelligent AI or not. (We already know how to solve the problem of world-hunger. People starve, not because we don't know how to grow enough food or transport it, but because of malice and neglect: they starve either because somebody wants them dead, or because others don't care enough to feed them.) There's only so many new Hummers we can build before the environment collapses, and us with it, and the AI probably knows exactly how many. (Well, to the nearest thousand, at least.)

So the AI would probably keep a small breeding population of a few hundred million, neuter the rest of us, and in a few decades our population would be sustainable again. Assuming it let four or five billion of us die before introducing all those miracle cures for cancer, aging and similar.

On the other hand, if the AI was focused on individuals, rather than the long-term safety of the population, things could get even more messy. Imagine having an army of robots controlled by the AI telling us what to do, and what not to do, for the precisely calculated optimum happiness every moment of the day. Sit up straight, relax, eat your vegetables, stop gossiping on Facebook, no more sugar for you today, you haven't had your opiates yet, slow down, speed up, pick up your things, turn the radio down, turn it up, pay attention, chill out...

We'd go mad.

That's what I mean when I say some problems are intractable: they don't have an answer. You can't optimize human happiness, because the process of optimizing it changes the parameters of the problem. And the more the AI is capable of, the more resentful and angry people will be when something unexpected hits. Why didn't you stop the earthquake? Why did you let granddad choke on a fish bone?

Answer 15

I can't see any answer concerning Art, the only thing that distinct us from machine. Although there are technologies that allow Machine to create art out of some random generated algorithm but that only pre-defined algorithm not the real ability to create Art from emotion and feeling.

And besides, even if the ASI solve all of humanity problem, one still find interest in doing things just for the sake of doing it. Even ASI can provide me food for daily need, I still want to grow my own vegetable garden just for the sake of growing it.

Hand-made thing still have it valuable. Even ASI can create for me a beautiful wooden table, some still prefer one made by his own hand. And the fun in trading it for other man made furniture will not be decrease by the fact that ASI can clone the exact same one for you.

The life of humanity will be much easier, but they will still can find interest in their life. I suppose.

Answer 16

Mass sterilization. The A.I. would determine there are too many people and make it so that only a fraction of the population could have kids.