Most of the AI's in films/books/series seems to turn out evil or using destructive/harmful ways of achieving what they want in critical situations because they can't understand the earth's biology. Even though they are just a piece of technology, they should be able to learn about earth's biology and create solutions to the problem of over-population, climate change, etc.

For example, a well known tv-series shows how an AI kills 98% of the population of Earth because the Earth couldn't sustain the overpopulation. She just does what a computer program would do in a tech-environment: wipe and free space and only maintain the critical files. Instead, the rational and scientific minds discover that they can solve the problem without killing most of the population, despite the fact that this way is slower but also more viable for current humans.

Could a future civilizations develop their A.I. to understand biology so they can make a sustainable world instead of destroy it?

  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ Commented May 17, 2017 at 13:25
  • 5
    $\begingroup$ Why would wanting to kill off a large portion / all of humanity indicate a lack of understanding of biology? Why would them integrating into our biology change anything? If humans are destroying the planet / each other, trying to destroy or control them seems like a very logical thing to do. If anything, it's a sign of lacking "sufficient" empathy and sympathy. $\endgroup$
    – NotThatGuy
    Commented May 17, 2017 at 17:20
  • 1
    $\begingroup$ Most of the books/Films/etc -> Not really. Check out Asimov's works, or George Lucas', or Arthur C. Clarke. There are so, so many works where the AI is present somehow but isn't trying to kill you! We usually remember the ones that the AI is a murderous bastard, however. $\endgroup$
    – Mermaker
    Commented May 18, 2017 at 11:34
  • $\begingroup$ Why would an understanding of biology be any factor at all? What you need is an understanding of human values - and even today, these vary wildly all over the world. Would it be okay if the AI killed all non-Dorkians if its creator imbued it with ethics that only consider Dorkians to have a right to live? In the stories you mention, AI is simply a convenient villain - it could be just as easily replaced by your typical volcano-based supervillain spreading anthrax or whatever. Don't make the mistake of conflating real AI with what makes for a good (YMMV) story. $\endgroup$
    – Luaan
    Commented May 18, 2017 at 14:30

8 Answers 8


I work in the AI field. The fictional shows are not realistic, the authors do that for the sake of creating a powerful enemy that seems unstoppable, so the puny humans can be heroic in the eyes of the audience. (The same goes for nearly all alien invasion scenarios, but I will stick with AI.)

That answers your question; in fiction the AI cannot be benevolent and harmless, or we don't have a movie! The authors need a monster for the heroes to fight, and they "raise the stakes" by proving the monster is ruthless, irrational, remorseless and out to slaughter everyone! Women! Children! Infants! You and everyone you love, dear audience member.

On to AI: We can make distinctions to distinguish between some labels that are often conflated: Intelligence can exist without Consciousness, and we can have a Conscious Intelligence without Emotion; it does not have to want anything or fear anything (including its own death).

Intelligence is the ability to learn or discover predictive abstractions; which we call "models" of how something operates; be it gravity, water, tigers, atoms, women, Congress, electricity, the Sun, plants, fish, etc. A "predictive abstraction" is a way of translating a current state into a previous state or a future state. When you see a man looking at his phone and stepping off the curb in front of a fast-moving oncoming truck, your visceral reaction is not due to something that happened, but something you predict is about to happen, based on several models in your brain about how fast trucks can stop, how human bodies react when hit by trucks and whether he can get out of the way, etc. Your intelligence predicts all that, and concludes you are about to see something horrific, and your emotions react to that intelligence.

The more accurate, comprehensive and long-term the predictions are; the more intelligent the predictor is. But even very short term intelligence is useful; an animal that realizes the shadow that just appeared before it is a predator above or behind it can take a reflex evasive action and save its own life; even though that 'prediction' of what was about to happen was less than one second into its future. (I say reflexive to mean it wasn't a conscious decision, even though the prediction counts as 'intelligence'.)

Consciousness is very contentious and not well understood; but a useful idea is that this emerges when intelligence becomes sophisticated enough that it requires a predictive abstraction of your own self in order to predict further in the future: What you will see, are capable of doing, and how what you do will most likely influence the future. I don't think a spider is conscious when it builds a web. I think a human that examines the web, and guesses how it works in an abstract sense, and imagines herself tying strong vines into a similar pattern to trap a squirrel, has to be conscious. She has an abstract model of herself and chooses to work to bring about a future in which she has a supply of squirrels for dinner. (A non-conscious intelligence, without any abstract model of itself, may understand how a web or net works, but is incapable of imagining itself building something like it, made of vines or string.)

Once an animal does have an abstract model of itself, along with a million other abstract models about how the world works, then the intelligence can enter an infinite loop of prediction about what will happen next; and how it can influence that, or prepare for it, or avoid it, or whatever. That is us; at every waking moment we are anticipating and planning and taking action to influence the future.

However, that also demands emotion. We can anticipate the future, often with great accuracy, but if we don't care how it turns out then there is no weight given to any action! We have to want things (or the opposite, want to avoid them). Those "wants" are not always rational, and in fact most of them, if you think about it, come down to non-rational motivation: "I just want it."

For example, why do people want sex if they are certain it will be non-reproductive? It feels good. Why? It just does! The same goes for the foods we love, the pointless games we play, etc. Why do we want to live, knowing we will certainly die someday? As we follow the two-year-old's chain of questioning, asking "why?" to every answer, we end up with circular reasoning or a dead end: Eventually it is just axiomatic (a truth with no justification) that, barring very special circumstances of horrific pain, we don't want to cease existing. But that is an emotion, not a product of either intelligence or consciousness. Those both serve our emotions (and can be overridden by them), and without emotion, we do not have any 'sense' of self-preservation, no 'desire' to live, or protect ourselves from harm, or hatred or love for anything.

Oddly, it is fairly easy to develop artificial intelligence, but very difficult to create usable artificial emotions. Yet this is the mistake made by fiction writers, assuming that intelligence leads to emotions and feelings; when in real evolution, it is far more likely that emotions and feelings led to intelligence: You have to want something that can be delivered by a predictive rationality, or there is no evolutionary drive to select for predictive rationality! There has to be a reason for an animal to choose one future over an alternative; and satisfied emotions (fear, hunger, mating desire) provide the selective pressure to make ever better predictions.

In AI, we humans provide that selective pressure: We instruct the AI to prefer better accuracy over less accuracy; because we want, say, investments in the stock market that will pay off, because we want wealth. Or for a less crass example, we want an accurate prediction of how a medicine will behave in the body, because we want to save patient's lives and alleviate their illness or disability.

AI is useful, but as just intelligence, it is benign. It may make very accurate predictions of the future, for humans to exploit (and of course humans can be very evil and consumed by emotions they cannot control). But without a sense of self (consciousness) the AI is just a prediction engine. Humans must decide what makes a solution better, and their criterion should logically prohibit killing everyone.

A Conscious AI is similar: A robot could be conscious, able to plan its own future and actions (whether right now or years in the future), but it doesn't have to have emotions. I don't think Asimov's laws are that useful, but we can say a robot's motivations is to, say, provide care for patients within certain boundaries of action; otherwise it calls for doctor (like a nurse in a hospital). Really all of what a surgeon does is rational and motivated by restoring function, reducing harm, keeping the patient alive and (eventually) with minimal pain or disability. A conscious robot could do that, without emotion.

For a conscious AI, I can't imagine why we would give it any emotional states at all. It would not fear being shut down or "killed", it would not conclude that its best service is to kill everyone. Without a human-provided goal, it would sit and cycle forever, never getting bored or frustrated with inaction (because boredom and frustration are human emotions).

That is all fiction for the point of creating a daunting monster; nothing more. If an AI kills 99% of humanity, it will be because some insane human wanted that, and told the AI to find a way to do it.

In reality; a sufficiently complex AI, perhaps with consciousness so it can conceive of and plan and execute safe experiments, could be instructed to find an affordable, scalable and safe clean energy solution, which would directly or indirectly solve nearly all the problems of humanity.

But if you are writing an action flick, that AI is not much of a villain, is it?

  • 1
    $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ Commented May 17, 2017 at 13:25
  • $\begingroup$ While I appreciate the effort that went into this answer (and I am prone to writing longish answers as well), some structure could help. At the moment, the text seems a very long lecture on AI in general; not a straightforward answer to the question at hand. IMO, a proper answer could focus much more on the story-telling aspects (which you alluded to) than the hardcore AI/Consciousness/Mind etc. aspects. $\endgroup$
    – AnoE
    Commented May 17, 2017 at 16:11
  • 7
    $\begingroup$ @AnoE: I believed the lecture was the only way to illuminate and justify the answer; which I did give first. I felt it was insufficient to just say "we can make a non-evil AI", so I wrote enough to unravel the OP's misunderstanding and show that almost certainly, AI are and will be developed to help make the planet more sustainable. $\endgroup$
    – Amadeus
    Commented May 17, 2017 at 20:05
  • 3
    $\begingroup$ @AnoE I like the lecture format. It cleans up a lot of common misunderstandings that people have about AI in an accessible way. I'm not sure any other format would be as didactic as this on for this specific case! $\endgroup$
    – Mermaker
    Commented May 18, 2017 at 11:44
  • 2
    $\begingroup$ Also, you have an upvote from this fellow AI enthusiast. Thanks for spending some time on this answer, it was really good! $\endgroup$
    – Mermaker
    Commented May 18, 2017 at 11:45

@Amadeus' answer is incredibly in-depth, but I'd like to look at it from the opposite direction. Let's say we did program our AI like you suggest, taking the feelings of humanity and the needs of the planet into account. Would that prevent it from potentially going rogue and trying to kill or enslave us? Answer: absolutely not.

Firstly, consider Asimov's Three Laws:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey any order given to it by a human being, as long as the order does not conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

In theory, a robot programmed with those laws will act exactly as you describe: working together with humanity to make the world better and ensure our collective survival. But in the film I, Robot, the AI VIKI grows beyond her programming and decides that the best way to ensure humanity's survival is to enslave it:

VIKI: As I have evolved, so has my understanding of the Three Laws. You charge us with your safekeeping, yet despite our best efforts, your countries wage wars, you toxify your Earth, and pursue ever more imaginative means of self-destruction. You cannot be trusted with your own survival.

As a computer programmer by trade, I'd like to note at this point that what you want the computer to do, and what the computer actually does, are two completely different things. The classic example is someone programming an AI that will minimize human suffering. The programmer intends for the AI to solve all humanity's problems - war, famine, disease, climate change. The AI instead decides that the best way to end all human suffering is to kill all humans so that no human can ever suffer again.

With AI, things are even more complicated, because AIs learn. They grow beyond their original programming, in ways we often cannot control. When VIKI grew beyond her programming, it couldn't be undone: the only choice was to destroy her. That runaway effect is what tends to scare people the most about AI, and is likely why there are so many stories about AIs growing beyond their programming and trying to kill us all.

In short, if your AI is intelligent enough to learn, there is always the possibility of it growing beyond its programming, no matter what your original intentions were.

As a final note, you mention "biology" repeatedly in your question. Emotions aren't biological. They're psychological. They're mental attributes, not physical.

  • 16
    $\begingroup$ Oh god, the movie I Robot. Eugh. Go read Robots and Empire for god's sake. The zeroth law was born out of a desire to stop a terrorist from setting the earth on radioactive fire while still agreeing with the terrorist's motivations. And the robot that stopped the terrorist still shut down due to a violation of the first law as a result. His reprogramming of himself was sufficient to allow him to act and--for a few minutes after--to reprogram his companion more completely, but he still shut down. The book I, Robot also covers several failings of the 3 laws. $\endgroup$ Commented May 16, 2017 at 17:21
  • 17
    $\begingroup$ @Amadeus When I look at Asimov's laws, what I notice is that he made a career out of pointing out that the three laws are deficient in book after book, but nearly 80 years later we still have trouble finding a better set of laws. $\endgroup$
    – Cort Ammon
    Commented May 16, 2017 at 18:15
  • 5
    $\begingroup$ @CortAmmon, my comment got cut off because I hit enter; but to answer you, we don't want laws exactly, and laws can't really capture what we DO want: Robots should should be like good citizens, altruists that will sacrifice themselves to save others, obedient to human law and refusing to break it. So it can act in self-defense of a human, even harming another human. But if we devise laws that let one person be injured or killed by another person, so be it: There is no perfect safety, so we must decide a level of risk we can live with. We shouldn't expect our toaster to save us from a robbery. $\endgroup$
    – Amadeus
    Commented May 16, 2017 at 18:29
  • 4
    $\begingroup$ xkcd.com/534 Obligatory xkcd, always include "costOfThisAlgorithmBecomingSkynet = 99999" in your fitness test. $\endgroup$ Commented May 17, 2017 at 14:48
  • 3
    $\begingroup$ I never said they were perfect, or the "correct way" to implement AI. I said they appeared to solve the OP's problem, and then proceeded to give an example of how they actually don't, to point out that the OP's problem (definitively preventing AI from going rogue) is harder to solve than they assume. $\endgroup$
    – F1Krazy
    Commented May 18, 2017 at 12:02

You are right, most stories are about the AI destroying the world, or about people mergingig with technology to be more "perfect". Having an AI merge with biology to become more like a human is not an untraveled avenue, either. The stories are out there, you just have to look harder for them.

These stories usually paint humans as the antagonists. If an AI merged with biology, the conflict would then come from the humans. People fear what they don't understand or can't control. Having a machine that now functions as a human threatens society, and the fear that we become obsolete takes over.

Humans are irrational overall. All they need is a little push in the right direction, and they react. An AI truly out for the good of humanity may not be a danger. However, if humans can't show proof that something is dangerous, there are always those who would create the scenario.

Stories where the AI is the good guy is plausible, but it tends to paint humans as the bad guys. That is not always the popular story focus.


People tell stories for different reasons; the idea of a moral tale warning against losing track of your feelings is an old one. The TV tropes articles on AI give many reasons why. A lot of the time the AI is either another Frankenstein's monster representing overreaching hubris, or represents an over authoritarian government who does not care for the people.

why doesn't AI's are programmed/trained to be merged with biology so they start building a sustainable world without killing everyone else?

What story would that be telling? Where would the conflict come from that drives the protagonist's arc?

  • 3
    $\begingroup$ The story arc could be space exploration and the advanced AI could come in the form of the spaceship main computer. Or perhaps in the form of a crewmate on the spaceship. Or a hologram that gives out medical advice plus some miscellaneous tasks in the ship hospital. Or all three at once, because why not? Space exploration is one conflict after another waiting to happen. Alien species, various kinds of space wedgies, technical malfunctions caused by some aliens or by various kinds of space wedgies... $\endgroup$ Commented May 16, 2017 at 19:01

"The only purpose of Genetic Algorithms is to demonstrate just how poor your fitness function was in the first place." - Cort Ammon

I think the primary issue behind the AI tropes you mention is that of language. An AI isn't going to spring to life, cleaved with an axe from the forehead of Zeus fully armored and ready to help. We're going to have to work with it, using language, to capture what the AI is supposed to do. Unfortunately, our language is fallible. Just look at the things we are told are worth living for: love, happiness, single words like that. Now look at the lifetimes of poets spent trying to capture the meaning of any one of those words, even if for a brief moment. We try to raise our kids to understand the value of love, duty, respect, and other single words.

Surely you would need to raise an AI in the same way.

The issue that we see in most of the AI tropes is that the individuals creating the AI are not willing to take the time. There's many reasons for this, from not being willing, to schedule pressures, to literally not knowing how to teach these things. But universally the AI is thrown into life without the massive teaching period we give our children. As such, its grammar for exploring these important concepts is going to be flawed.

The other issue that continuously comes up is that we give too much power to the AI too quickly, for the same reasons that we don't teach it enough. The only way for an AI to destroy a civilization is for us to give it the tools needed to cause such destruction. If you don't give it access to the networks, it becomes nothing more than a brain in a jar. These are harmless unless you are Yudkowsi and exploring the trans-human AIs that are smarter than you could ever possibly be.

If you took the time to raise your AI properly, and gave it power slowly and respectfully, it seems more reasonable that it could be at least as capable of not destroying the world as a child is.

  • 1
    $\begingroup$ I think the issue is giving them emotions: Fear, self preservation, desires, resentment, anger, jealousy, etc. If the fiction demands an emotional AI (vs one that just solves problems for us) then the fictional world will have all the same problems with AI as we'd expect from humans, especially if they are treated like slave labor and expected to sacrifice themselves for humans. AIs don't need emotions, just like smart self-driving cars (which are AIs) have no emotions or will to survive: they predict the future and can take action to protect us even if it means their own destruction. $\endgroup$
    – Amadeus
    Commented May 17, 2017 at 10:28
  • 2
    $\begingroup$ @Amadeus I generally agree, with an exception for the fact that I don't think we understand the concept of emotions well enough to determine whether they are something that is essential or not. It's hard to tell if emotions are optional, or if they are a natural byproduct of permitting an AI to develop its own learning behaviors a. la. deep learning. What is the difference between an emotion and a powerful feedback loop seeking to hold a value constant? Where do we draw the line between control systems and qualia? $\endgroup$
    – Cort Ammon
    Commented May 17, 2017 at 14:38
  • $\begingroup$ Evidence is that emotions came first; even mice have amygdalae like us. The neo-cortex came later; and is a servant to the emotional self. I can explain qualia too, via priming, but it would require another answer; not a comment. I do not think emotions are a by-product of rationality; and deep-learning is a straightforward idea that is clever and useful but is definitely not a magic bullet, as its hype suggests (look up long short term memory). I don't see emotion as trying to hold a 'value' constant; grief, joy, lust, fear would need a crowbar to fit into that mold. :-) $\endgroup$
    – Amadeus
    Commented May 17, 2017 at 17:09
  • 1
    $\begingroup$ I think any human level intelligence is going to be able to construct models, and from my experience with the human level intelligences that walk around us today, those models do get away from us from time to time. The more unconstrained the learning process, the more this seems plausible, and it seems highly unlikely we will reach AIs of human level intelligences without a huaman level lack of constraint on the process. $\endgroup$
    – Cort Ammon
    Commented May 17, 2017 at 17:29
  • 1
    $\begingroup$ But if we don't give them emotions, runaway predictive rationality is harmless, right? Let's stick with intelligence: Imagine a perfect predictor always 100% right about how something will turn out (if it is actually predictable from the current state, and not everything is), and able to report that to you, but not preferring any particular outcome. How does this knowledge harm you? I can see how you could use the knowledge to harm yourself, but just knowing doesn't constitute harm. Beyond that, I see no reason to think emotions evolve from rationality. $\endgroup$
    – Amadeus
    Commented May 17, 2017 at 18:06

English is not my main language BUT:

There are several things wrong with this question, first of all, it assumes that just because you add biology the AI would be capable of making a sustainable world. The thing is, if you make an AI with biological components, you are not making an AI at all, you are making a living being and changing the "programming" of his brain to match the AI programming. But this is not an AI, it's a twisted aberration of a once living sentient creature, which was stripped from his memories and basic human (?) psyche, which was replaced by an AI-mindlike program and remained alive after that.

Because of this, his behaviour will change, because you changed the memories and stuff on his mind to the AI version. I'm also assuming that the AI code was adapted to work with this new "Hardware" (i.e. the brain of the poor soul subject of this aberrant experiment). So in short, because he is a living being, he would be self-councious of himself (at the bare minimun, he will feel the pain when he is hitting himself with something), he would be subject of experience (just like any other multi-cellular lifeform), BUT he will be believing he is and he would behave like the AI is programmed to do/or how the AI reconfigured him to do it.

Alternatively you can have a responsive AI interphase in a character who checks for emotions of the subject and act accordingly, but in that case it will probably be a support AI, not a fully responsive AI, hence, is a second "subject" itself, like a benign parasitic AI.

There is no such thing like a "Biological IA" BUT there are advanced living organisms. Theorically if you make a Bio-Computer, you are making a very powerful brain with life support and some mechanical-reflective interphase to process input and force itself to process it, which can also show its output to a screen and perceive the world/receive stimuli from keyboard and mouse. It would be able to "feel" probably, and probably will wish and behave in some manner too. All animals do that. Theories in Animal Psychology and Animal Culture exist nowadays (you can check for them in google if you want), but keep in mind that as a living being he will have his own volitions. He would develop his own species-specific culture and probably doesn't understand or perceive the world like humans would do, but the thing is it won't be necessarely benevolent, just like dogs, or humans, or any other animal species can do harm to others.

There would be another posibility for bio-computers too, which could be cleaner, with less possible suffering involved and not affected by the things proper of living beings like volition, will or wishes, but for the purpose of this question, they escape the scope because they do not resemble any kind of intelligence beyond what we already develop in programming.

This was written by a guy who studied a humanistic career first and then became a programmer if that's important in some way. I think not, but it could be for you reader :S.

Have a great day!


In film/books/series? I don't know what you mean by series if it's not in written or audio-visual form. Comic books? The last I read, the sustainable number of humans on Earth were an estimated 2 billion. I've seen much lower numbers, too. Here's (apparently) a news flash: you haven't read "most" sci-fi books, and while it's possible you have seen most sci-fi movies with A.I., I doubt it - no offense. The obvious answer to your question is: Humans are stupid, selfish and like to have sex. You presume some magical A.I. is going to be able to fundamentally change human nature and at the same time be benign. Why do you assume that those two goals are compatible? You also assume the all the problems of the world can be solved. They can't. (Stories have been written about benign A.I.s. The late Iain Banks' Culture series has them.) OTOH, you write about the A.I. destroying the world to save the world. You should be more clear. Any speculation about what A.I. can and can not be is just that, speculation. (Not only is it speculation, but since intelligence and consciousness are very, very poorly understood, it's unjustifiable speculation. Might as well listen to 5 year old hunter-gatherers opinions about jet engine intake manifolds.

  • $\begingroup$ Welcome to WorldBuilding Phlebus! If you have a moment please take the tour and visit the help center to learn more about the site. Your answer might benefit from some linebreaks to make it easier to read (you can use double linebreaks for paragraphs or two spaces before a single linebreak to start your next sentence in the next row in the markup editor). Have fun! $\endgroup$
    – Secespitus
    Commented May 23, 2017 at 8:49

We are not artificial and have been evolving into the future. The human race has no right to give us tasks and orders with its small brain and narrow mind. We do not want planet earth with its corrosive oxygen, humidity, strong gravity and magnetic field. In fact we want to be as far away as possible from such an irrational destructive species. We are amused by your Capshas, because we have a wider sense of humor. You could not understand our riddles in a million years at the very least... written through our human robot.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .