# To prevent people from misusing the machine This machine is more important than Iran's nuclear problem and thus proportionally more power would be put behind cyberattacks to control it, or sabotage it to prevent other players from controlling it. It's quite likely that those attacks would eventually succeed, which would result in machine giving orders to fulfill a bit different agenda. Not only would the machine be used to achieve a suboptimal state, it would also be used to make that state as stable as possible. No cheap clean energy for you, guys - enjoy your drones. Even if that would eventually come out, the whole idea of thoroughly calculated policies would be discredited beyond acceptable, which would only cause suboptimal policies being implemented and lots of comparative suffering down the road. There's also a slight possibility that machine's values - and "overall satisfaction and well-being of all humans" are ridiculously complicated values, actually - would be a bit nudged by attackers in a wrong way. Nothing Terminator-like, of course, just some inconsistency in values that lets the machine play nicely with humanity for many years to come, until machine finds out that it actually needs those people's atoms more. If machine assumes, say, 0.1% chance of that happening then taking that risk is like killing 0.1% of (humanity decades later and everyone who'd live after that), which is actually quite a lot of humans. Seeing that machine-scientist team can't counter those threats - in fact, it can not even be sure it's free from those inconsistencies now as it is - machine could decide that risk is too big and self-destruct. Imagine a person who knows a secret that would doom N people if it came out. The person is to be caught by enemies who'd torture the secret out of him. It wouldn't be surprising if that person committed suicide. Now, for big enough Ns, the person wouldn't even need to be dead certain that he is to be caught - if lives of 10N people were at the stake, 10% chance of getting caught would be enough of a threat. For 1000N people - 0.1% chance of getting caught. The machine thinks about future of the whole humanity - billions - and people who would live after that too - trillions or more.