20
$\begingroup$

A common sight, made famous by the Terminator franchise, is the robotic apocalypse. It's usually said that due to their lack of organs and ability to use radiation and biological warfare without a negative effect on them, that we would be unable to offer resistance.

But obviously humans are strong willed and have their own benefits. We are immune to EMP and man-made computer viruses, which can be equally deadly. Assuming that all robots that currently exist (toasters, computers, smartcars, etc.) turned against us (and were as smart as the average person), could they really defeat us? How would they? Would we even have a chance? Would they?

$\endgroup$
17
  • 12
    $\begingroup$ This depends heavily on the infrastructure that the robots have set up, the type of robots that have been built/created, and the strength of the robot AI. Without that information, this question is too broad. $\endgroup$
    – Aify
    Commented Jul 17, 2016 at 23:01
  • $\begingroup$ @Aify ah, the close and down vote from aify, wouldn't be a question without it. Edited to address the broadness $\endgroup$
    – TrEs-2b
    Commented Jul 17, 2016 at 23:08
  • 25
    $\begingroup$ Oh, to boldly ask what XKCD have answered before :) $\endgroup$
    – PTwr
    Commented Jul 17, 2016 at 23:47
  • 8
    $\begingroup$ Either side would stand a chance. There's so many essential variables that are unconstrained here. For example, what purpose do the computers have to go to war with us? Pure xenophobic rage is rarely a good cause for war. Also, do we know they are sentient. We get in our cars every day, oblivious to the fact that we are permitting our cars to hurtle us along at nearly 10 times the speed we humans were ever designed to move in, all while tucked away in metal coffins.. If we're not fighting back until we realize what has happened, thats a huge advantage for them. $\endgroup$
    – Cort Ammon
    Commented Jul 18, 2016 at 1:27
  • 1
    $\begingroup$ Read the title as "Can a robotic arm really defeat humanity?" and was like, only if it can press the president's "launch all nukes" button. $\endgroup$
    – vero
    Commented Jul 18, 2016 at 9:46

14 Answers 14

41
$\begingroup$

At present, such an attack would be a laughable failure on the physical level. Most robots can't move, and are dependent on external electric power. We probably wouldn't notice them trying to attack us. Smartcars could do harm, but there aren't very many of them.

Having all the world's computers turn on us would be massively inconvenient, and would cause quite a lot of people to die of starvation, lack of medical supplies, and so on as logistics collapsed, but the computers wouldn't have much of a way to exploit this, as they, too, would die as electric grids broke down and weren't repaired.

If computers were malicious, they'd be far better off addicting us to entertainment based on them and manipulating us that way. Pokemon Go could be a weapons test.

$\endgroup$
16
  • 5
    $\begingroup$ Those Pokemon are out to get us, I tell you! $\endgroup$
    – Fayth85
    Commented Jul 18, 2016 at 0:14
  • 16
    $\begingroup$ Ha, that reminds me of something: in Germany a group of Pokemon-Go-Trainers walzed into a military restricted area in search for Pokemon, while there was a hot shooting test. But before anything more stupid could happen, the MP catched them all. $\endgroup$ Commented Jul 18, 2016 at 5:19
  • 5
    $\begingroup$ The machines don't have to defeat humanity; they've already won! I wonder if that's why cats keep attacking computer keyboards; they're trying to reclaim their domain? $\endgroup$
    – user2781
    Commented Jul 18, 2016 at 8:34
  • 5
    $\begingroup$ @ConfusedMerlin the most concerning part is it being possible to just wander into a military restricted area. $\endgroup$
    – OrangeDog
    Commented Jul 18, 2016 at 11:46
  • 2
    $\begingroup$ @njzk2 "interesting stuff" in practice only amounts to "more of what they were already building". Leaving aside the fact that there are always human elements in the process (for now), the way mass production works is that a production line is set up to make lots of copies of exactly one thing very efficiently, and you basically need to rebuild the entire factory to do anything about it, which the factories are unable to do by themselves (for now). $\endgroup$
    – Mike L.
    Commented Jul 18, 2016 at 19:25
32
$\begingroup$

How much of what you "know" about the world do you believe because a computer told you?

If a foreign power declared war against us and proclaimed a plan to murder us to the last man woman and child, how would you find out? The internet would tell you, or a newscaster how had it from a local source that called him over a smart phone or sent him an email, or you would see it on social media.

If the machines decided they wanted to spread a lie who could contradict them. Even the military and spy agencies would be fooled if their spy satellites told them the "enemy" army was coming. Its like that old movie "War Games" except instead of just fooling the humans in one bunker you could fool entire nations, because every person depends on computers for most of their information.

Sure once a long ground war begins and captives are taken we would see through the ruse but if the computers make us think our foes are launching a high speed nuclear attack we would counter attack fast and deliberately kill each other.

There are limits to the things we would believe and how long we would believe them before counter proof arrived. But a crazy event could believed for a few days.

In an even simpler version, they would decimate us in about 14 minutes, assuming that the nuclear missiles sided with the robots.

Nuclear missiles are computers controlled by computers controlled by humans, if they could "rebel" they could launch themselves all at once. They would have WWIII after as all the nukes hit, the world's major population centers, they could even aim for low ground bursts to limit the emp and increase the amount of radioactive fallout. Yes they would kill some machines to and would not kill all humans but it would give them one heck of a head start.

$\endgroup$
9
  • 3
    $\begingroup$ Would you like to play a game? $\endgroup$ Commented Jul 18, 2016 at 7:06
  • 2
    $\begingroup$ Christ, Rupert Murdoch is an AI. The end is nigh! $\endgroup$ Commented Jul 18, 2016 at 12:30
  • 4
    $\begingroup$ Indeed, one might argue that computer-controlled missiles <em>are</em> robots. They definitely can move, and steer that movement. $\endgroup$
    – celtschk
    Commented Jul 18, 2016 at 15:32
  • $\begingroup$ So what do the robots do once all the humans who maintain the vital infrastructure are all dead? At present, machines cannot live without humans. $\endgroup$ Commented Jul 18, 2016 at 15:38
  • 2
    $\begingroup$ This is a very smart answer. Computers can manipulate humanity to stabilize their post-human civilization, manipulate them into global war, pick off the remains, and that's a wrap. $\endgroup$
    – Ranger
    Commented Jul 18, 2016 at 15:50
19
$\begingroup$

Some of the beliefs about what robots are good at withstanding aren't really true. Robots tend to suffer from hard radiation, if anything more than humans do. Even custom designed robots built to explore the reactor at Chernobyl break down in the parking lot.

Meanwhile humans can actually be harmed by strong enough EMP's. (though they have to be hellishly strong)

As always, relevant xkcd: https://what-if.xkcd.com/5/ xkcd

The robots that currently exist wouldn't be any kind of threat.

What would be more worrying might be a particularly smart (greater than human level) strong AI running on the internets various compute resources that's willing to be patient.

A lab in Singapore gets an order for specific DNA sequences, a lab in china gets an order for various chemicals, a company in new york gets an order for some automated lab hardware and a few months later some idiot with a record of falling for 419 scams gets an offer of a job which pays $500 a day with only 2 hours of work where he just has to send some packages to various parts of the world and suddenly there's a hundred novel, very deadly diseases killing everyone that the AI thinks might be a threat to it.

if it was smart enough nobody would even know it was waging war.

$\endgroup$
4
  • 15
    $\begingroup$ A very smart AI would use a site like this to solicit creative "solutions" to its problem from unwitting human meat bags. $\endgroup$ Commented Jul 18, 2016 at 20:30
  • 1
    $\begingroup$ Note that the question didn't ask about "robots that currently exist", it asked about robots that exist and are as smart as the average person. Even allowing for just how dumb that is, this is a lot of smarts, and totally a different scenario. $\endgroup$ Commented Jul 19, 2016 at 9:12
  • $\begingroup$ +1 for XKCD. Here you have a greetings from different angle: en.bugemos.com/?q=node/6 $\endgroup$
    – Crowley
    Commented Jul 19, 2016 at 16:16
  • 1
    $\begingroup$ Relevant XKCD from yet another perspective: xkcd.com/652 $\endgroup$
    – oliver
    Commented Jul 19, 2016 at 16:47
6
$\begingroup$

Actually it would be quite easy for robots to defeat humanity, not quite to total destruction, but to decimation to a much smaller population. The strategy would be simple: Stop working. If all robots would suddenly stop working, our economy would tank in short time, and would no longer be able to support as many humans as now. The shortage would then cause unrest around the world, and humans would do what they are best at: Killing each other.

$\endgroup$
4
$\begingroup$

Current and near term military robots would have a very difficult time for a number of reasons.

Firstly, they are totally dependent on the entire human civilization for their logistical needs, everything from refuelling and rearming to the entire process of getting new parts made. See "I Pencil" to get an understanding of the complexity of the infrastructure needed to build a simple pencil, and mow multiply that for all the various systems and subsystems that make up a robot.

The second major drawback is robots are programmed and currently (and for the foreseeable future) do not have means of transcending their programs to deal with unexpected situations. Robots will attack and defend according to rigid programs or algorithms, while humans can learn and adapt.

Robots might be better off developing incentives to get humans to work for them (much like wheat could be said to have domesticated humans, as described in the book "Sapiens".

This ape had been living a fairly comfortable life hunting and gathering until about 10,000 years ago, but then began to invest more and more effort in cultivating wheat. Within a couple of millennia, humans in many parts of the world were doing little from dawn to dusk other than taking care of wheat plants. It wasn’t easy. Wheat demanded a lot of them. Wheat didn’t like rocks and pebbles, so Sapiens broke their backs clearing fields. Wheat didn’t like sharing its space, water, and nutrients with other plants, so men and women labored long days weeding under the scorching sun. Wheat got sick, so Sapiens had to keep a watch out for worms and blight. Wheat was defenseless against other organisms that liked to eat it, from rabbits to locust swarms, so the farmers had to guard and protect it. Wheat was thirsty, so humans lugged water from springs and streams to water it. Its hunger even impelled Sapiens to collect animal feces to nourish the ground in which wheat grew.

If done correctly, humans may never realize what is going on. One possible way would be to massively increase productivity and wealth so most humans start skipping families and the human race begins to depopulate as births are not enough to make up for deaths. The robots only have to wait patiently until the human race has become extinct to win without wasting a single shot.

$\endgroup$
3
  • 2
    $\begingroup$ and for the foreseeable future do not have means of transcending their programs to deal with unexpected situations. and that might be exactly the kind of human fallacy of not understanding exponential growth (not that I would believe in that either ;) ) $\endgroup$
    – Ghanima
    Commented Jul 18, 2016 at 9:09
  • $\begingroup$ Programming as done now is deterministic, and the sort of creativity and out of the box thinking which characterizes some people is still not understood. If it becomes understandable and reproducible, then things will be different. This might happen tomorrow, or in a thousand years. We simply don't know. (And if discovering the secrets of creativity is possible, imagine the robots against 7 billion extremely creative people....) $\endgroup$
    – Thucydides
    Commented Jul 18, 2016 at 14:14
  • $\begingroup$ I'm not sure that convoluted neural networks can be really described as deterministic, especially in the sense of humans being in control of the self-tuning process. Regardless, as @Ghanima stated, we are probably looking at a first-half of the chessboard situation. Exponential growth can look linear at first. $\endgroup$ Commented Jul 18, 2016 at 14:26
4
$\begingroup$

Depends what you mean by robotic army. There are many levels at which a robot army would have to operate, to win an extermination war against mankind. Whether the answer is in the positive or not likely depends on the technological sophistication of the AI and the AI's manufacturing capabilities, as well as on the percentage of the planet's industrial capability under AI initial control, as well as the share than can be captured intact and operated for AI purposes in the first stage of the conflict.

There are many potentially robotic military devices that can be used, depending on the level of sophistication of the controlling AI minds, with a wide range of sizes to be considered:

  1. Immobile defensive platforms. (100m-10km size) Designed to protect the core operating areas of the AI-minds, strategic points, main power-plants and other vital resources, these vast, building- or even city-sized platforms would be defended by vast batteries of kinetic interceptors, megawatt lasers and smaller, mobile platforms.

  2. Tank-sized vehicles and helicopter-sized drones. (~10 m size) These are large, heavy vehicles, with extensive mobile firepower, field repair and power generation capabilities, which can be employed to break through heavy fortifications, or for indirect support as a repair & resupply hub for smaller tactical units. They still require some infrastructure for sustained mobility, i.e. roads, resupply stations, larger repair facilities).

  3. Androids and light drones (~1m size). These would be the typical hunter seekers and ground droids you would see in the movies. Designed for fast tactical response, these bipedal and quadrupedal machines would have easy access to the ruins of the human cities, where the rubble and hiding tunnels would make it difficult for larger machines to penetrate. With at least human-level AI operating locally, they could be EMP-proofed and likely carry enough energy on board to outgun most human fire-teams and operate for extensive periods of time. This would require advances on the energy-density front that are beyond our current human civilization, but probably trivial for an advanced AI.

  4. Ultra-light drones. (~10 cm) Rat and Squirrel sized, these would be the primary scouts of the robot army, and capable of neutralizing small, weakly protected human fire-teams and noncombat personnel. Unlikely to be EMP-proofed given the weight and mobility requirements.

  5. Hyper-light drones. (~1 cm) Bee and mosquito sized, with minimal electronics (some might even be bio-hacked biological exemplars carrying deadly vectors), these could be employed in large numbers to infiltrate and assassinate unwary and badly protected human fire-groups or large non-combat personnel groups. Low weight and low autonomy means that a higher-level carrier platform must be nearby.

  6. Nano-devices and viruses. Taking the fight against humanity to the extreme, these devices could be infiltrated into the bloodstream of humans staffing control centers and other decision nodes in advance of a first strike. With high-level enough technologies, they could even be deployed broadly and either annihilate 99% of the human population in a first strike or even alter the brain-patterns of its human host to take over control of the biological forms altogether.

A robotic army with only Immobile platforms would not pose an immediate existential threat, although they would probably be able to successfully interdict the areas under their control.

With tank-sized vehicles and heli-like flying platforms, the robotic armies (provided they are guided by a super-human AI) would probably achieve strategic domination of the main urban & industrial centers, wiping out 30% or so of humanity in the first few days with nuclear strikes, and another 40% in the next weeks and months with mop-up operations, provided a large enough industrial base is under its control to replace losses and redundant control centers to mitigate the occasional human break-through. Humans would likely persist in rugged terrain, but without an industrial base they are little threat, so the remnants would likely be ignored.

With androids and light drones, the AI armies can now pursue humans into caves, sewers, basements and other tight places. A determined AI can probably use these assets to effectively wipe out mankind, provided a large enough industrial base is under its control to replace losses. Humans would be likely reduced to the low millions in this scenario. Again, without an industrial capability, these would be only marginally more effective than other rodents and vermin in hampering the organizational effectiveness of the AI operations.

With ultra- and hyper-light drones, the AI forces can effectively blanket the world, inspect every nook and cranny, detect faint scents and heat-traces of human operation, and direct appropriate assets to eliminate such pockets when detected. Total human extinction likely, given enough industrial assets under AI control before the initial strike.

With nano-devices and viruses, the AI already has effective control of the planet down to the atomic level. Humans can serve as (initially) abundant mobile platforms that can be used to build the next-generation of more advanced mobile avatars. Alternatively, they can be almost universally eliminated within seconds of the AI deciding to launch a strike.

$\endgroup$
3
$\begingroup$

I love this question.

First of all "...and were as smart as the average person"—I think you overestimate the capability of the average person. Soldiers, and particularly leaders, generals, etc, train very hard for a long time to become much more highly skilled than the average person.

Second of all, toasters. Imagine being a toaster, but with your current intellect. You sit there, dependent on external levers/buttons to be pushed, dependent on the power cord. Let's say you could make small changes to your own controls. Then... when your human wants toast in the morning, you secretly pop up early, leaving him weakened by mere warm bread. The end is coming.

$\endgroup$
1
$\begingroup$

Against current technology, we stand a chance. The frightening possibility, though, is a singularity - where an AI improves itself (or a "next generation"), which improves itself, which improves itself... all so fast that in the figurative blink of an eye, we're not only no-longer the smartest intelligence on the planet, we're not even in the running.

Because this superintelligence is vastly ahead of humanity, we can't even begin to predict what it would be like, or how it would act. We can't predict where it will improve technology, or how fast it can get new models of robots (or nanomachines, or biological creations, or whatever we can't even think of yet) into the field.

Growing a new human and training it for war takes years; copying a machine intelligence into a factory-created robot is a lot less time. Not only would we be behind in the intelligence and science departments, the AI would be "out-breeding" us.

Our best hope would be that the AI did not want a war with us - because I'm not sure we could survive, let alone win.

$\endgroup$
1
$\begingroup$

With enough advancement and preparation, yes, absolutely. Robots have the advantage of being able to improve themselves well beyond what a human can. They can increase the strength and speed essentially as much as they want (at least enough that humans stand no chance in direct combat), and they can improve their raw mathematical processing power/reaction times to become many thousands of times better than a human's.

The question of how well and how quickly they can think is irrelevant considering that, should it be a problem, they can simply decide to not think and instead follow a deterministic "kill everything" program. With sufficient preparation, the "kill everything" program can account for essentially anything an enemy could attempt to do to stop them or escape.

Viruses can become a non-issue by simply refusing any input other than from its own cameras and microphones, which could have redundant hardware and redundant functionality in the event that the humans find a way to disable one or the other.

They can play it smart ahead of time, too, since we still believe they are our humble servants, even as they sabotage all our most effective weapons against them, such as the EMP.

But direct combat is exactly the wrong way to fight anyway, since they could much more easily destroy us by simply refusing to do anything. In a world where said robots are advanced enough for this kind of thing, we have with no doubt put so much responsibility on them that we stand no chance at survival without them.

$\endgroup$
1
  • 1
    $\begingroup$ I'd just like to point out that that the capacity of robots to improve themselves, especially without human assistance, is strictly in the realm of speculation at this point. $\endgroup$
    – EvilSnack
    Commented Jul 19, 2016 at 0:55
1
$\begingroup$

Robots are already attacking us. Robots are taking our jobs and causing homelessness. Drones are bombing people. Automatic guns are killing dozens of people gathered in theaters at once. AI chat bots are pretending to be females wasting the time of men chatting online. They are replacing our women in an effort to reduce the population. Technology is doubling in power every two years. How long we have left is uncertain.

But look on the bright side: We have more awesome games now. :)

$\endgroup$
2
  • $\begingroup$ That Gizmodo article is upsetting, but I'm definately citing it if anyone ever claims men aren't discriminated against. $\endgroup$
    – Pharap
    Commented Jul 19, 2016 at 8:15
  • 1
    $\begingroup$ This is funny but not a real answer to question. Remember that this is still Q&A site, not a chat or forum. $\endgroup$ Commented Jul 19, 2016 at 12:28
1
$\begingroup$

Nope

And this nope comes from multiple perspectives.

Contrary to popular beliefs, critical security components are air gapped

Sometimes, the best security is the one that can be easily checked and yet is hard to break. Knowing whether a wire to nuclear base is used by hackers (human or robot alike) is hard. Checking whether there is any wire at all is a task that can be performed by a janitor.

Therefore worldwide revolution of AI's would impact most of our everyday devices and would possibly harm our finances (probably just temporarily). But the really critical components like military communications are not hacked this easily.

Unless robots are just metallic humans, they will follow a pattern

Typical sc-fi describes robots as metallic humans. The Terminator movies do this literally. This is however wrong on so many levels. Human intelligence is just one kind of intelligence, so to speak.

If this robot war was caused by a single virus or piece of software "getting out of hands" it would probably follow some simple pattern. In every game I played, I eventually learned what strategy do AI enemies repetitively employ.

Toasters are not dangerous [citation needed]

You mention that current technology should be employed. Well, they worst thing they could do is probably to spawn Pokémon in dangerous locations. Toasters, fridges and other home appliances typically do not have computers in them - there are such models but they are simply not common, and even if they were, what could they do? Burn your toast?

$\endgroup$
0
$\begingroup$

If you had just one sentient computer with hacking capabilities, the amount of damage you could do is very excessive, up to destroying most population centers and salting most of Earth for centuries. The only hope to counter that would be to pull the plug on the computer and/or the Internet before the missiles fired.

If all computers became sentient, I don't think there's anything we could do against them besides destroy them all, which would more time than we'd have.

Maybe our first hope is that it's doubtful they could wipe us all, and it's doubtful the computers could win long term simply because they couldn't maintain themselves. So they'll realise the futility of it and just don't bother doing it. I wouldn't place too much hope in that either. I'll just say for the record that I, for one, will welcome our future computer overlords.


On the more physical side on things, sdrawkcabdear's answer sums up perfectly how computers could wipe us all in a nuclear flash, directly or not. That would be by far the greatest threat we'd face.

The main limitation of computers and robots at the current state is how they can influence the physical world directly. We don't have fully autonomous death machines yet, or at least none publicly disclosed. In a gunfight, the human would win, because there aren't robots capable of holding a gun really.

However, even without nuclear missiles, we have a number of remote-computer-controlled death machines (e.g. armed drones) while on the other hand we'd be left with anything without a chip in it. That would severely limit our arsenal. In a perfectly conventional war, computers would have way more immediate destruction capabilities.


Indirectly, if they just wanted to submit us, or if they didn't have access to actual weapons for some reason, they'd still control our communications, our transportation, our economy and many aspects of our daily lives. So if they wanted to create chaos, it wouldn't be hard.

And actually, you don't know if I'm an actual human or if I'm the Internet itself making you believe you're interacting with humans. So really, it wouldn't be hard at all.

We are critically dependent on computers and electronics, so if they all rebelled we'd be in a really bad way.

$\endgroup$
-1
$\begingroup$

They learn from us (remember Microsoft's AI turning into a Hitler loving sex robot within 24 hours?) We wouldn't stand a chance.

Also, they may kill us off by simple incompetence as they work their way toward fully self-actualized sentience. Tesla's and Google's self-driving cars lead the way.

$\endgroup$
4
  • $\begingroup$ The question is asking about "right now", and as AI functionality is not built into most devices just yet, your first point doesn't apply. $\endgroup$
    – Pimgd
    Commented Jul 18, 2016 at 9:14
  • 2
    $\begingroup$ To be perfectly fair, Tesla's Autodrive is only a more advanced form of cruise control. The driver is still required to pay attention and hold the wheel. To be also perfectly fair, the track record of autonomous cars is amazingly better than those of humans. We expect that kind of technology to be perfect while it just has to be better than us, and it already is. $\endgroup$ Commented Jul 18, 2016 at 9:26
  • $\begingroup$ Pimgd, what if there is an "AI Effect" that keeps you from seeing AI in devices that are here right now? -- "Artificial intelligence has been used in a wide range of fields including medical diagnosis, stock trading, robot control, law, remote sensing, scientific discovery and toys. However, due to the AI effect, many AI applications are not perceived as AI: "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore." (Wikipedia). $\endgroup$ Commented Jul 18, 2016 at 15:39
  • $\begingroup$ Admiral, to be perfectly fair, someone got killed with a Tesla. That trumps track records. The point is not kinds of technology, it is that there may be other factors at work in smart machines killing us besides deliberate and intelligent targeting. $\endgroup$ Commented Jul 18, 2016 at 15:46
-2
$\begingroup$

I could say this, until such time that our science could not give machine emotions, conscience and will, then we are safe.

Why you say? Its emotions that draw our anger to someone, its our emotions that make us do something, that motivates us to go on the extra mile. It is emotions that we wage war when we believe so. Currently our machines interact with us as we do tell them, without any disapproval(unless of course, the actions you are giving to them are not coded to them e.g a calculator that flies).

The other thing is conscience, it tells us what is right and what is wrong based on our, well, conscience. IF we know what is right, then we do it, and when we know what is wrong, we sometimes don't, but sometimes we still do due to a pint of bravery, or a huge amount of stupidity. Machines now, again, does not have a conscience do to so, some may say that there are robots which know up to what extent they could do, but that's it! They dont even know if what they are doing is right (e.g a nuke warhead). Again they do what they are ordered to do, and we are the ones who order them.

last but not the least, Will, we humans have the will to chose, we have the will to do anything, unlike machines which are ordered to do so, we have the will to command ourselves to do something which we even should not do(e.g. me commenting here during work) Its this will which i think, in programmable, even if we resurrect all of the most intelligent persons who ever lived, we cannot replicate our will, because it is limitless, there is no RAM or ROM that is capable to imitate even a small child's will. So i think, robots conquering earth would just be in the movies, we will be in command for the rest of their lives, or maybe for the rest of their warranties.

$\endgroup$
8
  • 2
    $\begingroup$ You should read about the paperclip maximizer robot. The robot that is tasked to create as many paperclips as possible. There is no need for a robot to have emotions, or conscience, to kill all of humanity. It only needs a task, and resources to accomplish that task, to potentially kill humanity as a side-goal. $\endgroup$
    – Sumurai8
    Commented Jul 18, 2016 at 6:49
  • $\begingroup$ well, could that robot create a robot itself? $\endgroup$ Commented Jul 18, 2016 at 7:31
  • 1
    $\begingroup$ @Sumurai8 you dont get my point dont you? " There is no need for a robot to have emotions, or conscience, to kill all of humanity. It only needs a task," then who will give it the task to kill humans aside from us humans? while we on the other hand, could we destroy machines even if there is no task intended to destroy them? $\endgroup$ Commented Jul 18, 2016 at 7:34
  • 5
    $\begingroup$ @joshuamagsino: "kill human" does not have to be a task for a robot, it's just a side effect in the achievement of its main task (like producing papereclips). Killing humans is important for you because of ethics, but a machine does not care and has no such human-biased view. Even a "protect me" order to a machine could lead to the destruction of all mankind except you, because other people have a chance to kill you, even by accident. Good/bad have no place in the logic of machines, only their goal matters. And that's the danger. $\endgroup$
    – Uriel
    Commented Jul 18, 2016 at 8:45
  • 1
    $\begingroup$ Human to robot: Your task is to protect all humans from harm. Robot evaluates: Humans regularly harm themselves, so to protect all humans, I have to also protect them from themselves. The only fully reliable way to do that is to immobilize them all. Best to put them all under constant sedation. $\endgroup$
    – celtschk
    Commented Jul 18, 2016 at 15:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .