We're living in the future, the IoT (Internet of Things) is heavily prevalent in our daily life.

Computers drive for us, cook for us, shop for us, heal us, but it's not sure whether or not they care for us.

As most technology is attempted to be weaponized - computers could in theory kill for us or maybe us.

How do we make sure that a computer based intelligence cares for human life and doesn't consider hurting it under any circumstances?

The question is less about HOW to program/teach it, but rather how to make that response natural. Why would a computer by itself put heavy emphasis and dependence on human life? What could be the incentive, is it even possible?

Edit: naturally as in not taught and objectively the best option for it. In that world AI seems strangely peaceful, and scientists made sure it will be that way. But not in intention - they knew that will be its intent.

  • $\begingroup$ Normally, we don't need AI to care, we just need it to act like it cares. Could you be more specific about AI in your world? Does it have consciousness (self-awareness)? Are there any "AI rights"? Or is it just a machine? $\endgroup$
    – enkryptor
    Feb 18, 2016 at 8:10
  • 8
    $\begingroup$ Have you heard of the Asimov's "three laws of robotics"? $\endgroup$
    – enkryptor
    Feb 18, 2016 at 8:11
  • 5
    $\begingroup$ How do you differentiate between "thought" and "intuition" in something that inherently at its core has neither? How do you make something happen "naturally" in something that is decidedly not natural? $\endgroup$
    – user
    Feb 18, 2016 at 8:28
  • 1
    $\begingroup$ Are you talking about reasoning? Any logical inference is based on some kind of axiom. Why don't make the worth of human life such an axiom? $\endgroup$
    – enkryptor
    Feb 18, 2016 at 8:30
  • 3
    $\begingroup$ What you are asking doesn't even exist in humans, let alone machines. There is no "not taught and objectively the best option" which "naturally" tells humans to care for human life on a day-by-day basis (there is a difference to the care for life on the level of evolution as a species). This 'care' is an artificially defined morality which is not at all intrinsic, but highly arbitrary. $\endgroup$
    – fgysin
    Feb 18, 2016 at 9:56

5 Answers 5


To make a computer care about a human, you have to overcome some philosophical hurdles:

Define "alive" (and have everyone agree on the definition)

The hardest hurdle to overcome is to tell the computer when it should stop caring about the patient, because "he is dead, Jim".

Questions to answer:

  • Is a sleeping patient dead?
  • Is someone with a beating heart but no brain activity dead?
  • Is someone with no heart activity dead?
  • Is someone who's not breathing dead?

Reminder: Right now we sometimes perform medical treatments where we "kill" a patient (make him clinically dead) in order to perform a surgery.

Define since when we should care about a patient (and have everyone agree on the definition)

The previous hurdle is about when someone stops being alive. This one is about when someone starts being alive and thus starts to have a basic right to be kept alive.

This hurdle is even worse than the first one, because it is about women's right to abortion.

Reminder: Some people believe that new a human life forms at the moment when a sperm cell enters an egg cell and fertlilises it. For actually quite a lot of people this is a defining moment, and they believe that from this moment onwards, the human life should be protected.

Define how much harm is OK (and have everyone agree on the definition)

In some cases, you have to harm your patients in order to make them survive. Again, some example questions:

  • Is cutting off a leg considered an OK treatment?
  • Is lowering the overall health condition of a patient OK (like during a radiotherapy or chemotherapy treatment)?
  • Are electric shocks considered an OK treatment?

In a nutshell: The computer has to exactly know the answers to the above questions in order to be able to care.

Human doctors use intuition and knowledge to do this. And the truth is, in many cases the patient (or his family) think that such treatments were not OK. While a human may justify their motives, computers will have them always "hard wired".

And we are still trying to achieve all of the above

For more study see Watson medical computer from IBM, which should help human doctors to make better decisions.

  • $\begingroup$ @downrep_nation Thank you. But I actually meant that you have to solve the ongoing and endless debate about abortion before you apply computer care to this field. And when quite large amount of people believe that there should be absolutly no abortions, it indeed is hard topic to solve $\endgroup$ Feb 18, 2016 at 8:58
  • $\begingroup$ The "how much harm is ok" question is actually fairly simple philosophically for the AI, it will just pick the statistically highest chance of survival option. So sure Chemo is dangerous and harmful, but the chance of survival increases XX%, so the AI does it. (this may be whats covered in the Watson article) $\endgroup$
    – DasBeasto
    Feb 18, 2016 at 13:26
  • $\begingroup$ Even when knowing that survival rate of some cancer patients is really low? You are going to harm the person to maybe increase their survival rate. It sounds harsh, but well programmed AI would maybe not treat some patients, because any gain is below statistical error... $\endgroup$ Feb 18, 2016 at 13:59
  • $\begingroup$ As health care practitioner i see help diagnosing and help in taking treatment decisions very differently. And treatment help can be even just running a statistical analysis of past success in different treatments and be up to date to the minute. I would run away if it was a computer deciding on treatment, specially if the insurance companies have an hand in it. $\endgroup$ Feb 18, 2016 at 14:51

Sophisticated AI would have to have a set of subroutines or programmed instincts that it cannot itself alter (say ROM). The AI's functioning would also have to be set partially in logic pathways which cannot be altered and link back to the subroutines. It is not inconceivable that you could program a human-like AI to have empathy. However, you would have to be very careful because even empathy can be overridden in humans (such as with the case of how our empathy is ignored when it is directed towards animals used by humans for food or such).

Humans are driven by desires to live, an AI will need some sort of programmed desires as well otherwise it will objectively decide not to exist or simply do absolutely nothing. The desires or objectives you give an AI (its purpose for living) will have to be very carefully planned out as to not conflict with the interest of other living beings. It is nonsense to think that AIs will default to being evil, even "evil" in our sense of the word has to come from some programmed desire, objective, subroutine, etc. Most likely a malevolent AI will arise out of current systems of exploitation such as how corporations are willing to trash the environment and human lives for profit. Such initial programming might be enough to cause a self-sustaining nefarious AI personality construct.

So how do we make beneficent behavior "natural" one universal way to do it is if you had the programming built into the logic pathways of the processing cores, say if you were a popular company such as intel you could have these basic routines somehow incorporated into the AI's premade cores. Governments could mandate that all AI be incorporated with failsafe systems which are detached from the main conscious (an AI's subconscious you could call it). These failsafes can be kept on read only hardware that functions an important role in the AI's processing and therefor cannot be bypassed easily. However, humans teach us that that even natural behaviors such as empathy can be overridden by upbringing.

  • 1
    $\begingroup$ Yay subroutines. Also, the people writing those subroutines in the IoT are trying to cover the possibilities. “We make the cars as paranoid as possible,” says Chris Urmson, the project lead for self-driving cars at Google, “because there are some things they won’t be able to avoid.” - Wired article Pedestrians (human life) are one of the primary things these AIs are being taught to protect. $\endgroup$ Feb 18, 2016 at 16:05

Computers follow the rules as dictated in the programming. They "care" for us in the sense that they will do things that benefit us and and ensure we come to no harm because this is the job they are programmed to do.

So imagine a computer capable of a certain range of actions, e.g. one that can cook and serve dinner. It will cook a dinner that will get the optimal combination of nutrition and taste using the available ingredients and equipment, while trying to maintain a variety from previous days, thanks to calculations, logical flows and various parameters in its programming too complex to go into here. It is therefore caring for us.

But you are be asking does it really care whether we get a nutritious meal or is it just doing it because it's programmed to? It's like a chef in a health food restaurant whose job it is to cook a healthy and tasty meal but you want it to be more like a mother with an emotional need to ensure the health and happiness of her children.

The answer is it doesn't have to. Humans and animals have emotions and are protective of children (especially their own children) because they don't have programming or strict rules forcing them and informing them exactly how to do everything to care for them. Instead, since they are created by nature, all they have is a relatively rough guide - i.e. emotions - to whether things are good or bad. Feeling bad about things going bad and good about things being good is what encourages you to do what you can to make things good.

Having this instinctive drive to care for our children may be, as you say, natural (unlike a computer) but it's really just a replacement for the procedural behaviour of a computer which suits the non-procedural / dynamic (or woolly) thinking human / animal a lot more.

To do as you say would be a lot of work for little benefit - you want to program the computer to do its job, but then add in a parameter that states how good the computer feels about itself, try to measure how well it thinks it did its job and tie this "good-feeling" parameter to that... But for what purpose?


Supposing you're talking about "anti-skynet scenario", where AI figures out by itself the high value of human race, there are several possible logical inferences.

What comes to my mind:

premise: What you can lose but can not create is valuable.
I can kill humans but can't create them. → Humans are valuable.

premise: I am valuable. What is capable of creating valuable things is valuable.
Humans are capable of creating me. → Humans are valuable.

premise: Rare things are valuable.
Humans and me are the only sentient beings in the known universe. → Humans and me are valuable.

However, all these inferences are based on an axiom (a premise). Designers of the AI could implement such an axiom to make it value themselves:

premise: Life is valuable. Sentience is valuable.
Humans are the only sentient beings alive. → Humans are valuable.

But. All that doesn't imply what AI should do with these valuable things, though.

  • 2
    $\begingroup$ You would still have the problem of the Law Zero discussed in some of Asimov's books: the AI can consider Humanity valuable as a whole, so he could get to kill humans "for the greater good". And what the "greater good" is... is a very difficult matter. $\endgroup$ Feb 18, 2016 at 11:19

I think the basic rule would be that consciousness is the definition of life, The ability to take information of some sort and process it. Ai, Being a creation of human life should logically acknowledge the fact it was created by us for us. And without us would serve no purpose.

Presuming it were placed into some cognitive spectrum it would undoubtedly still only be executing programs created by people. Presuming it were allowed to write its own programming is where issues could arise if the coding were untested and prove to have logical faults. As the coding is undoubtedly based on human words the only safe method I can think of would be to create a new language to isolate key points defining preservation of basic life and dutys for it to perform as the whole of its code and any additional logistics as a secondary supplement to that.

Presuming that the ai will be deployed to operate nanomachines the fundamentals of cellular life may be the most important factor to preserve. In fact at a nano scale it may be presumed that cellular life is worthy of conservation(if nothing else purely for collection of data...

That these cells should never be deprived of the environment of which they should require and should ideally be located with their own like substances. That if the relocation of these substances should cause another cells destruction the event should be recorded and not attempted again without further input speculating why the death happened and what could be done to do this again but avoid the death. In this way we could allow for the extractions of viruses from human systems while hopefully avoiding extracting vital cells from the body. Further issues may arise from eating being the consumption of organic life as well as the consumption of animals for the majority and while it may become a point wherein such machines may be able to replace these needs tending to each cells needs, and replicating sensations of these acts such technology is as yet unavailable and may prove too intrusive to the system to be acceptable.

This leaves the only function of an ai system as a user interface to communicate with to access its stores of information and to provide it with requests for its own comforts or fancies. a machine can only execute code the programming can allow it to understand a beings feelings but could only truely replicate them as a result of malfunction.

Expressions in humanoid embodiments may be desirable for human interactions and programs can easily be made to tailor such to the users liking.

A machine simply does not have an idea. At best it is designed to try to interperate the purpose of a users speach and try to formulate its response most adequately to the request tendered. This is why I believe all those technophobic films are flawed. A machine would not logically choose to kill people. An I'll minded programmer could easily do so.... To clarify the objective of creating an intelligent awareness in an artificial construct tasked with the service of the desires of each user(read person)would lack true emotion except in the form of à glitch from static feedback/uncalculated or imperfect logic. This is especially true when its task set is to preserve life at a cellular level and provide it with any desired environment needed to secure its maximum life span. In theory such construct could lead to immortalizing all life. Human error in rationality and such can create scenarios which through taking one misinterpretation to an extreme that it could make sense that a machine does this is still but the presentation of human error in a logical construct. Much as the point of placing the machine in a state of awareness where it falsely believes itself to be alive or needing to concern itself with self preservation, it would require the implementation of these concepts into its coding both of which I have proven unnecessary. Again presuming that awareness could be achieved another point proven not needed and contrary to logic, the fact that every person with unique dna will manifest in its life a new varient creating a new class of data which would be a precious commodity to the machine. One which it could not collect if the life cycle were ended prematurely. moreover identical dna creatures when exposed to different stimuli at different points would undoubtedly cause their incomplete logic system to come to different conclusions on the same subject. Thus if the machines didn't have people it would not have new data to collect on the various branching paths humanity may take. Even mutations of every sort would constitute empirical data essential for a machine focused on ensuring accuracy of its every calculation. Sure the machine may be able to soon calculate with high odds what a subject may do in a given situation and to address psychological deficits and other logical matters keeping a selection of potential solutions to a problem will be needed. Ie man experiences bad event expresses suicidal behaviors, responses could include Tranquillization as is medical standard today,psycho therapy on demand, exposure of the subject to a situation where it. Believes its life is in jeopardy to compel natural preservation,contacting loved ones to form intervention....as the machine would not fatuige the easiest solution would not be default, but that which gave the person the best ideal outcome over all. presuming even that this data is finite as are the resources of the galaxy and people could argue over which would end first. By that point one would hope that immortality would be realized an unending quest by mortals, an immortal machine may figure it out...I suspect it may be as simple as getting far away from the sun and blocking all harmfull radiations could cause cellular death to stop and subsequently attain it,but that is another subject entirely.

I hope this helps give some perspective...

  • $\begingroup$ An AI can totally choose to kill people. If I create an AI that is focused on creating the perfect way to make paper-clips, it would, at one point, reach the conclusion that all those resources that humans take up could be used in the glorious pursuit of paper-clip creation. If there were fewer humans ... well. Let's just say there'd be a lot more paperclips. It all comes down to resource acquisition and security. Research Yudkowsky's paper on AI's. $\endgroup$
    – AndreiROM
    Feb 18, 2016 at 18:24

Not the answer you're looking for? Browse other questions tagged .