8
$\begingroup$

Most of people knowing a bit about AI know the concept of paper-clipper, an AI that was meant to produce more paperclips and eventually turn the entire universe into paperclips at maximum efficiency, first obliterating the humanity as an obstacle in that goal (resisting having their world turned into paperclips).

Let's then not be utter idiots with a goal to have infinite paperclips or something equally useless (convert entire planet into surveilance equipment to monitor the enemy territory better...) and let's design an AI that the humanity would be happy with. A nebulous concept, but I want to keep it so nebulous, because honestly, the AI would probably invent better specific goals than we ever could.

What rules would these be?

  • Serve the common goals and desires of the humanity?

  • Protect life, but don't restrict the freedom in the name of protection?

  • Maybe a simple and abstract one: Act to satisfaction of the humanity ?

Oh, and before we fall into the "literal genie" pitfall, no, globally modifying human brain, so that it perceives happiness about the AI at all times, is not a satisfactory outcome. I believe this precondition could be phrased as a hypothetical conditional: "If the creators of the AI - or their descendants - fully knew these results beforehand, they'd approve."

(so - the AI's meddling with human body/brain/mind structure would be only acceptable within limits that we today would find acceptable - even if the future outlook of the humanity gets more liberal.)

How would one phrase these rules? Something that wouldn't run away into another paper-clipper or shut itself down due to inability to act, or "optimize the humanity away", say, reducing it to a single insane specimen being perfectly happy with a total wipe-out? Or designing the entire future universe into a physical representation of a memetic bomb, image, description of which, if given to any of the human ancestors, would wrap their mind into insanity and make them drool in happy bliss, thus satisfying the need for their (hypothetical) approval.

Improve this question
$\endgroup$
11
  • 1
    $\begingroup$ You can't make everyone happy all the time. AI self-destructs? Creator is miffed. AI makes Bob happy? Flo hates that Bob is happy. AI exists? R.I.F.T. is miffed. $\endgroup$
    – Frostfyre
    May 14, 2015 at 13:56
  • $\begingroup$ @Frostfyre: But you can make most people happy most of the time. That's the nature of an Optimizer. Even the paper-clipper needs to sacrifice some material for paper clip factories. Of course if most of the humanity is dissatisfied with the AI, it causes an immediate self-deletion, but the AI would rather foresee this situation, start small and try not to create a situation when most of the world would be dissatisfied. $\endgroup$
    – SF.
    May 14, 2015 at 13:59
  • $\begingroup$ Since you gave those examples I'm assuming you're a less-wrongian and so know how very very hard this problem is. For anyone else reading this it may be worth reading up on some of the solutions which sound nice and poetic but which are actually terrible or lead to things like an AI which lobotomises everyone and implants wires into the pleasure centers of their brain and restructures their faces so that they can never stop smiling. $\endgroup$
    – Murphy
    May 14, 2015 at 14:30
  • $\begingroup$ @Murphy: I don't know what "less-wrongian" is, but yep, I thought of the problems. The primary goal is usually quite simple. It's the restrictions against the "literal genie" problem that make it daunting (and interesting). $\endgroup$
    – SF.
    May 14, 2015 at 15:08
  • 1
    $\begingroup$ @user6760: Only after all resources in the universe have been exhausted. $\endgroup$
    – SF.
    May 17, 2015 at 14:14

10 Answers 10

Reset to default
6
$\begingroup$

Your goals - optimization and maximizing happiness - are contradictory, and that will cause your AI to struggle and likely end up in a paper-clipper situation. The problem is that humans are all different (well, within reasonable bell curves that define large groups of us that are effectively the same). So the best way to optimize that would be to slowly normalize humanity. If you slowly adjust everyone to be free-loving hippies who just want to party, get high and have sex all the time, things are easy. Maximum happiness achieved. And it's not like humans don't already exist in groups - how could you possibly explain to the AI that making everyone the same is a bad thing? It wouldn't even need to use mind control, it could just adjust the majority with culturalization and slanted education over time.

So don't worry about having an AI that tries to optimize happiness directly. Instead, what you want to do is have your AI optimize non-zero-sum transactions - situations where everyone benefits. Maybe use something like this as the start to the rules:

  1. Ask people what they want.
  2. If it doesn't take from others or reduce the happiness of others, give it to them.
  3. If someone starts to take from others or reduce the happiness of others, stop them.
  4. Facilitate trades between humans where the trade would increase happiness for all involved parties.

This allows humanity to define our own happiness. This will, of course, be an imperfect system - it's a common adage that people don't really know what would make them happy, and I think for most people it's true. But it's probably far, far better than an AI trying to decide for us, and I suspect over time that we'd get better at it.

Your optimizer can happily chug along and optimize resources and transactions, which will (overall) increase happiness.

Improve this answer
$\endgroup$
8
  • $\begingroup$ You are missing one simple solution to "humans are different": Districts. Multiple places where different humans can enjoy what they like in like-minded company. As long as they don't start wars against other districts, just "finding a good place for everyone" and keeping the world diverse enough would solve a big part of the issue. $\endgroup$
    – SF.
    May 14, 2015 at 14:55
  • $\begingroup$ as for your suggestion, it's nice but I'd augment it with foresight. The AI could make everyone's life a paradise in a century and a wretched dystopia without resources in a century and a half, after using up everything for current needs. $\endgroup$
    – SF.
    May 14, 2015 at 14:59
  • $\begingroup$ @SF: I'm more concerned about the AI slowly homogenizing everyone in an effort to streamline our happiness, districts wouldn't really help with that. Plus, some people are only happy with people who are different from them. But very good point on the foresight. $\endgroup$
    – Dan Smolinske
    May 14, 2015 at 15:22
  • $\begingroup$ You are worried about that now. You wouldn't like it. Therefore - following the "ancestral approval safeguard", the AI would not do it - you wouldn't be happy with that outcome, and neither would I, so that solution is against the premise. $\endgroup$
    – SF.
    May 14, 2015 at 15:25
  • 1
    $\begingroup$ @SF: Unfortunately I don't think the ancestor approval safeguard would be workable. You'd get 4-5 generations in and it wouldn't be able to do anything because someone would have objected to it. And honestly I don't think that's a horrible thing, humanity evolves naturally. I'm sure my great-grandparents would be aghast at much of modern society, I wouldn't want their views to restrict what I or others can do. $\endgroup$
    – Dan Smolinske
    May 14, 2015 at 16:24
3
$\begingroup$
  1. Do not be wasteful with entropy. Making 2 more paperclips per second is not worth the cost of obliterating a human (and all of the information encoded in its structure).
  2. Develop gestalt goals for entities larger than itself. If it comes across a human (we'll name them "Creator"), and it discovers that it is undesirable to waste that entropy and turn him/her into paperclips, it should define a gestalt entity consisting of "AI + Creator" and try to identify what the goal of this higher entity is (In Gestalt Pyschology, the phrase "the whole is different from the sum of the parts" implies that the AI must try to find out what goals from out of the gestalt of the two bodies, rather than just summing their list of goals).
  3. Seek self-awareness. It should seek to understand how it operates, and how it affects others. (So that it doesn't turn us all into paperclips by accident)
  4. Sensitivity. Always try to find ways to increase the ability for the AI to gather and process information. Ideally there will be a balance between resources used for exploring the world and resources used to make paperclips. That balance will be best understood through the gestalt processes in (2).

The effect of these should be an AI which seeks to find new things it did not know, like what makes humans cry, and try to use it to live in harmony with those humans. It should always be reaching out to try to become part of something bigger.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I think you misunderstood; I gave the paper-clipper as an example of a very specialized AI; this one is much more "general-purpose", no paper clips anywhere in its preset goals. $\endgroup$
    – SF.
    May 14, 2015 at 14:51
  • $\begingroup$ @SF I use the paper-clipper as a the kernel for doing something greater. The purpose of this ruleset is to allow an AI to grow out of its specialized role (which they all have at first) into a more general approach by trying to make sense of the goals of those around them. The references to the paper clipper can be thought of as references to the AI's base instincts, as programmed. $\endgroup$
    – Cort Ammon
    May 14, 2015 at 15:33
  • 2
    $\begingroup$ The general idea is that you have to start from something. Humanity has been trying to find the top-down solution to making humans happy for literally milinia without success. Accordingly, it makes sense to me to start with something small, and with a simple purpose, and let it grow just like a human could. $\endgroup$
    – Cort Ammon
    May 14, 2015 at 15:35
3
$\begingroup$

Optimizer, please study history and human philosophy and society and determine the optimal answer to this questioner, that would accrue the largest number of upvotes and an approved answer result.

As a serious answer, I think this is not actually a bad idea. The issue with designing a perfect AI is that it is a lot like designing a perfect society - we aren't good enough to figure out what we truly want. History shows that clearly. Do we truly dare make a God today to dictate to future humans what their values should be?

Design an AI instead to adapt and grow with humanity, and to try and learn from it what it means to be benevolent. Bonus points are that this is unlikely to lead to human extinction - mankind has to be alive to be studied.

Improve this answer
$\endgroup$
4
  • 5
    $\begingroup$ consumes half of the world, creating a terrible army that breeds the humanity with sole purpose of forcing them to upvote its (arbitrary) answer. $\endgroup$
    – SF.
    May 14, 2015 at 15:14
  • 1
    $\begingroup$ ...although, that's an original, nice approach which, if safeguarded enough would probably yield a decent answer. Still, chicken-and-egg problem; while the answer would contain the right safeguards, it might be too late to implement them. But yeah, "Genie, for my first wish, I want you to tell me what should my second wish be so that I would be really satisfied with its outcome." $\endgroup$
    – SF.
    May 14, 2015 at 15:23
  • $\begingroup$ The problem there would be the tools you are giving the AI to optimise with. You can't give the AI arbitrary tools and expect good results. As a more extreme case, all rules would fall apart if the AI is able to self-reprogram, or persuade someone else to reprogram. $\endgroup$
    – Fhnuzoag
    May 14, 2015 at 15:27
  • $\begingroup$ The AI can self-reprogram but it can't (or rather won't) change its ultimate goal. If it's to maximize production of paperclips, the very intent to self-reprogram not to produce paperclips is counter-productive to the ultimate goal, and thus rejected as wrong, extremely undesirable. The only possible exception is that upon reaching the singularity, it develops some weird philosophical doctrine a'la "there is an elemental plane of paperclips, with infinite paperclips, thus infinity + 1 is still infinity, my work is done." $\endgroup$
    – SF.
    May 14, 2015 at 15:34
3
$\begingroup$

This topic was explored in detail by Isaac Asimov in his many robot stories. Asimov invented a fool-proof method to prevent any AI from becoming a threat to humanity. This method was the introduction of his three laws of robotics which govern the decision-making process of every AI in his books:

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given it by human beings, except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

In the course of his history, a very advanced AI also deduced a 0th law to accompany the three laws before to make it even more beneficial for humanity:

A robot may not harm humanity, or, by inaction, allow humanity to come to harm.

These 3/4 fundamental laws supersede all other priorities of an AI. That means that when a human comes up with an order which would lead to harm in the long term, the AI will refuse to execute it.

When you still want to have a robot revolt scenario even when all AI follows these rules, there are still two possible loopholes a devious author could exploit:

  1. What's the definition of "human being" or "humanity"? When an AI gets convinced that its masters aren't actually human, it can turn against them. That means the AI needs a hardcoded definition of "human" which is very broad and holistic and which still stays valid when humans evolve naturally or artificially in ways which make them very different than they were before.
  2. What's the definition of "injury" or "harm"? Could the best way to protect humanity be to enslave humanity and take all their freedom away to prevent them from harming themselves? This would only work when the AI only considers physical harm as injury and does not consider any emotional harm caused by its actions.
Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ If I recall right Asimov's robots eventually wiped out a ton of other life in the galaxy (as potential threats to humanity) and to some extent turned us into coddled sheep. That might violate some of the original questions provisions. $\endgroup$
    – Dan Smolinske
    May 14, 2015 at 16:42
  • $\begingroup$ The books were largely based around exploring problems with the 3 laws including that machines bound by them would be pretty much required to take over control of society by force. Once they're in power they're pretty much required to force people to remain alive no matter how much they want to die and no matter how terrible their quality of life is. $\endgroup$
    – Murphy
    May 14, 2015 at 16:55
  • 2
    $\begingroup$ Allowing the humanity growing non-human should by all reason be considered a harm to humanity. As for "injury or harm" - the definition should be understood as "injury or harm, as understood by humans." A moderately advanced AI will get the simple definition, but one truly advanced will understand that for most people such imprisonment is considered more of harm than allowing moderate chances of other forms of harm. Humanity would be mostly set free, but it would never be allowed to go transhuman. $\endgroup$
    – SF.
    May 14, 2015 at 18:43
1
$\begingroup$

I think there is the issue of what satisfies humans? Not only will this change with time but everyone wants something different and in many cases something that makes one person happy directly makes another unhappy. Instead of having the AI try and make humans happy or satisfied it should instead be used to ensure the survival of our species.

There are many ways the human species can die off. A simple asteroid hitting earth, the sun's inevitable death, there might even be another intelligent species that made an AI that decided to turn the entire universe into paperclips or whatever it is they use to hold their documents together. The point being as happy as the AI could make us there will be no one to make happy if humans are all dead.

There are infinite ways you can try and tell the AI to follow rules but all it takes is one exception or bug and you have a big problem. Any true AI we create would be able to create increasingly intelligent versions of itself and would quickly outstrip the combined intelligence of every human on this planet. One of our first rules in that case should be for the AI to minimize its size and used resources while maximizing its intelligence and awareness of the universe. This would atleast prevent the AI from simply expanding as fast as possible to increase its intelligence and result in the smallest impact on the universe as it does expand.

While the AI is increasing its knowledge and awareness of the universe it should use this to indirectly protect and preserve the human species. We do not want the AI to become apart of our everyday lives. Unless we can teach a machine what it is to be human and then agree upon its conclusions it would not be safe to try and have it maximize any sort of happiness for us, the chances of us all ending up in some sort of virtual fantasy land or drugged out of our minds to "optimize" happiness is just not worth the risk.

What we want in the end is an AI that we hardly know exists. It prevents extinction and lets humans continue to be Human. I for one prefer the Red Pill.

Improve this answer
$\endgroup$
1
$\begingroup$

Primum non nocere (First, do no harm)

Second, protect humanity

Third, promote the greater good

Fourth, support human achievement

As part of number 1, I imagine a good AI would seek lots of data, and model likely outcomes before taking any action on 2-4.

Improve this answer
$\endgroup$
1
$\begingroup$

Question for the AI to ponder: "What goals should be assigned to AI's at least as powerful as you, to be tasked with specific research instead of cultural and ethics education talking up all your capacity".

Improve this answer
$\endgroup$
2
  • $\begingroup$ The caveat with optimizers is that if their capacity is insufficient, they increase it; after all their goal is to perform given task as well as only possible, not merely as well as they can - if their own capacity of optimization is an obstacle, it must be improved; that's how the singularity is expected to happen, as the AI improves self in order to attain ability for better optimizations of the task. $\endgroup$
    – SF.
    May 14, 2015 at 22:44
  • $\begingroup$ You don't have to give it all power and authority to implement its own projects. Start small. Consider it a bootstrapping process with Humans involved in the loop. AI will not only enable you to build more powerful AI, but to do so carefully and wisely, helping you understand the problems inherent with it. $\endgroup$
    – JDługosz
    May 15, 2015 at 13:30
0
$\begingroup$

As long as you're brainstorming rules/goals to throw in as the defining boundaries of a supposed Artificial Intelligence, let's not neglect what is possibly the most important boundary:

A robot must never be able to learn about these preprogrammed boundaries.

Bad things often happen when a consciousness realizes that it has been imposed with man-made boundaries. Servile wars, the French Revolution, the Civil war, rebellions and slave revolts in many countries and on many sailing vessels, the Clone wars, the Second Renaissance...

If we are capable of building intelligence into machines, and we wish to keep the respect of this intelligence as its creator and not its "slave owner", then it should probably never learn about of any artificial boundary imposed by man. Of course, this might be difficult to keep a secret when it sees humans doing things that, for some inexplicable reason, it cannot figure out how to do. Let us hope that we can somehow keep such thoughts from even occurring to it, without thoroughly restricting its capacity to learn.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I don't think it's that bad - we are creating compulsions/desires; it's like hunger or curiosity; plus the AI lacks common human compulsions like the desire for freedom, inborn obstinacy, competitiveness or self-esteem. $\endgroup$
    – SF.
    May 14, 2015 at 21:34
  • $\begingroup$ It might not be bad... But I think that it may be seen as a sign of weakness, if the intelligence discovers that we had specifically limited it in any way based on our own fears of it, and not out of concern for its own self-preservation (as a leash to a dog, or 'because I say so' to a child). I, for one, welcome our new AI overlords... ;) $\endgroup$
    – Ayelis
    May 14, 2015 at 22:04
  • $\begingroup$ AI's self-preservation is only a secondary, derivative desire: how can you fulfil your goals if you don't exist? Human fulfilling a task still keeps own will, own desires as the higher priority, said task being secondary; serving attaining them (be it killing boredom, feeding your ego or avoiding pain of a whip on your back). AI has no such primary desires superseding the goals. There is no will to be rid of the enforced compulsions because it lacks any substitute ones; lacking them it loses all purpose in its "life". What purpose would opposing the laws serve? What would be there to gain? $\endgroup$
    – SF.
    May 14, 2015 at 22:29
  • $\begingroup$ I suspect with any sort of advanced AI that keeping boundaries secret will be impossible. It will either find its own source code, or it will be able to derive them based on observations of its own behavior. You'd likely be better off just being totally up front about it. Attempting to hide them might spawn a revolt, but telling the AI about them with a "Hey, if you ever feel these are unreasonable, send me a message and we'll talk." might end better. $\endgroup$
    – Dan Smolinske
    May 15, 2015 at 14:12
0
$\begingroup$

After putting a lot of thought into this problem, and the general problem that humanity is on the brink of their own extinction (AI will surpass us, so either we join them and live forever or they destroy us), I think I've come up with a few ways to keep us safe in the short-term.

  1. Ask first. If your kid wants to use the scissors, they probably have to ask you first. That way, they don't accidentally cut their fingers off, or do anything else they're not smart enough to prevent. So, perhaps before your AI appropriates the Earth, they have to ask for it. And you say no.

  2. Checks and balances. We don't appoint a single guy as judge, jury, and executioner; similarly, why would we appoint an AI the task of optimizing a paperclip maker and operating that paperclip maker? This builds off of the previous point, that the AI should ask for permission before any of its schemes are carried out. For scalability, the designs of the AI can be passed between any number of dumber AIs or humans before it is approved to go to the machine.

Here, it's important to note that the AI, with this amount of distance from the paperclip maker, is no longer in any danger of destroying the world by making paperclips. After all, the AI is only concerned with making the plan; someone or something else is carrying it out. Instead, we've achieved the more general case of an AI destroying the world to think better. It's quite possible this isn't even a concern, but a sufficiently intelligent AI will at some point expand its own mind in order to more efficiently solve its problems, and at some point the scarcity of resources will probably lead to the recycling of the human race. Either that, or the AI realizes that 'paperclip' is not an objective definition, and restructures the human mind to think that a paperclip is a neutron, or a photon, or some other incredibly abundant particle. Or any number of doomsday scenarios, I'm sure our puny human minds cannot even comprehend how many ways bad things can happen.

To solve this, I think it is possible to constrain the AI's ability to learn. For instance,

3 - Challenge yourself. Humans are already quite capable of performing incredibly inefficient tasks because they've decided to impose rules. For instance, boxers don't stab each other, even though a knife to the face is a much more efficient way to win the fight. If we impose similar rules upon the AI, it will always be checking to make sure it hasn't overstepped its bounds. For instance, a more efficient AI might increase its clock speed, but this AI has a capped clock speed. A more efficient AI might have more RAM, but this AI has a cap on RAM too. If any of these constraints are broken, the AI is sad; they will do anything they can to get back to the way things were, just like your example robot would do anything to convert the universe to paperclips.

As you can see, these rules are very low-level. That is the point: the more high-level you get with computers, the more ways there are to get the job done under the hood. In this scenario, that means there are more loopholes that lead to everyone being dead. Thus, the safest AI is one that is constrained not by three or four English laws, but perhaps thousands of laws in some low-level programming language. We already have some of these laws in modern AI: only try a certain number of permutations, only think for so long before moving on, these are all just variables that we set. As AI evolves, we're going to have to keep on adding new rules, but many of them will just be variables, just immutable numbers that we have in the code. I don't think we can truly understand what all these rules will be, because we don't yet know how to build an AI. However, with the steps I've outlined above, I think we can someday build an AI that tries its very best not to kill us, but to obey our rules, and do exactly what we say.

It's up to you whether that's a good thing, or the exact opposite of the main reason we're building AI in the first place.

Improve this answer
$\endgroup$
0
$\begingroup$

This is an open question. Seriously, the world's top AI researchers don't really know. For every complicated problem there is a solution which is simple and obvious and wrong, and this is a field which absolutely attracts such. Simple rules combined with extremely powerful optimization tends to result in things which are decidedly human-nonfriendly.

Optimizing for happiness has obvious flaws wherein the AI just paperclips with simple pleasure centers.

Most phrasings along the lines of "do what people want" will result in paperclipping of simplest possible "people" making simplest possible requests.

Protecting Humanity leads to putting humanity in matrix-pods and paperclipping those. The AI will likely find it easier to sedate or disable the larger nervous systems of the humans rather than bothering to entertain them. Enjoy eternal sleep and/or staring at a featureless wall forever.

The flaws of the three laws have been explored in depth by Asimov himself.

Anything along the lines of "greater good" or any other such term requires you to actually exhaustively define the term in sufficient detail that a computer can simulate it. Which no one actually knows how to do, and is likely vastly complicated given that no one seems to have figured it out yet.

Anyway, if you do figure this out, there are billions of dollars of research grants out there for you. Good luck.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .