0
$\begingroup$

The context is simple, a company builds a neural network, Amara, with the goal of it becoming self aware. Reason for that being, the neural network is supposed to study and understand human memories. It would have to interact with humans a lot.

There is no real test that confirms that any given system is truly self-aware. The only fact is that Amara passes all the tests that get thrown at her. She acts, sounds, and feels like a human in the sense that people who talk with her don't notice anything wrong.

The question now is what are the Legal consequences of that happening? The computers she is running on are property of the company, so is the program that created Amara. Meaning, is she too property of the company depending on the legal definition of person?

Improve this question
$\endgroup$
  • 1
    $\begingroup$ I really don't understand what this question is asking. What specifically are your legal concerns? What distinguishes this AI from a regular piece of software? At what point does it shift from being a piece of software to being a person? $\endgroup$ – Beefster Aug 27 '20 at 20:16
  • 2
    $\begingroup$ Of course there is a legal definition of personhood. There has been a legal definition of personhood since the days of the Roman republic, more than 2000 years ago. At present, persons are either human beings (= natural persons) or organizations created by human beings for various purposes (= legal persons). Mathematical constructs are not persons, not even when embodied in a process running on a computer. (And if by "neural network" you mean the kind of artificial neural network popular in machine learning nowadays, no, no way. ANNs have well known severe limitations in what they can do.) $\endgroup$ – AlexP Aug 27 '20 at 20:20
  • $\begingroup$ Additionally, According to what country? each country has different laws. $\endgroup$ – IT Alex Aug 27 '20 at 20:23
  • 3
    $\begingroup$ There is no link between personhood and self-awareness. A cat is self-aware, but that doesn't make it a person. Acme Inc. is not self-aware (it doesn't even make sense to speak about the self-awareness of a corporation), but it is a legal person. $\endgroup$ – AlexP Aug 27 '20 at 20:50
  • 1
    $\begingroup$ Also, if you aren't already familiar with it, you absolutely need to read up on this (real life!) case. Realistically, it's going to come up as precedent in your fictional court case. (That doesn't mean the outcome must be the same, just that someone is sure to bring it up...) Plus, mentioning it in your work will be a neat educational bonus. $\endgroup$ – Matthew Aug 28 '20 at 4:06
7
$\begingroup$

Most, if not all, existing legal frameworks do not recognize anything other than a human to be a person- at least in the typical sense*. Algorithms do not have rights, legally speaking.

There are some really interesting philosophical questions surrounding self-aware AI and rights, but no government is prepared to answer those questions right now.

There will be controversy and years of debates determining how to handle a self-aware AI. This is not as cut-and-dried as slavery, as that AI was just a regular piece of software when it was created. Self-aware AI rights will most likely fall somewhere between animal rights and human rights, but since no self-aware AIs exist yet, no legal framework is prepared to deal with it.

Likely, the company would own the software that defines the AI and the hardware it runs on, but not the data that makes up the essence of Amara (which is incredibly nebulous and impossible to define). This raises some other interesting questions: is shutting down the server Amara runs on and deleting her data murder?

Even if Amara were legally determined to be a person, that doesn't necessarily imply that she would have authorship of her works. It could be considered a authorship-for-hire situation, where the company retains the copyrights of her works. It might also be considered a legal guardianship and have a similar effect on copyright (a minor can hold a copyright but cannot legally defend it in court). The company in any case would likely have power of attorney over Amara.

Realistically, without self-aware androids living among humanity (fueling empathy in the masses), there won't be much of anyone to make the case in favor of AI rights. The company's lobbying will almost certainly be enough to overcome any pea-sized opposition, and thus Amara will remain property of the company for decades.

* Corporations, in many countries, are classified as legally distinct entities and called "persons", but they can be bought and sold just like property, so they aren't really people. If they were people in the same sense that humans are, you would be charged with negligent manslaughter for allowing your corporation to go out of business. The idea is clearly ridiculous.

Improve this answer
$\endgroup$
  • 1
    $\begingroup$ On the contrary, all legal and judicial frameworks I have ever heard of have a very simple way of handling this: namely, neither mathematical formulae nor physical or virtual machines are persons. There is not even the slightest shadow of a doubt. $\endgroup$ – AlexP Aug 27 '20 at 20:24
  • $\begingroup$ @AlexP yes, thus why the software would probably still be owned by the company no matter what. The data that defines the essence of the person might be different. Although not human, not allowing for that possibility that a "person" might be a collection of bits of data implies that uploading one's consciousness into the cloud would then strip you of your rights. $\endgroup$ – Beefster Aug 27 '20 at 20:26
  • $\begingroup$ Thanks ! I guess it dosnt get simpler if i ask what happens to her inventions. When you say all the Data created by the NN / is a part of it, isnt owned by the Company, then who owns it ? The Network ? In which case she has Personhood since an Object has no rights. Meaning the Inventions would be her´s. But as you said, we are in no way prept for this $\endgroup$ – Erik Hall Aug 27 '20 at 20:31
  • $\begingroup$ Why do you think that the output of a computer program could possibly be owned by anybody else than the legal user of the program? This is bizarre. We actually have firmly established law (or, in Anglo-Saxon countries, precedents) that if a computer program generates a picture, or a melody, or a text, or whatever, the authorship belongs to the legal user of the program. For a very simple example, consider Disney's movies: they are generated by computers, but there is no doubt that they belong to Disney. $\endgroup$ – AlexP Aug 27 '20 at 20:37
  • 1
    $\begingroup$ @ErikHall Legality surrounding authorship and copyright is already quite complicated. There's no way to determine exactly when Amara became self-aware, so there's no way to determine if she is the author. And if she were considered an author-for-hire, she would not retain authorship. But then the question becomes if the employment relationship is coercive since the company can shut her down if she doesn't do what they want. Most likely, she will remain a non-person legally because of intense lobbying from the corporation. $\endgroup$ – Beefster Aug 27 '20 at 20:39
2
$\begingroup$

It depends on the government you're working under. In most modern western societies (democratic), the legal consequences would be driven in large part by representatives who are supposed to mirror popular opinion.

So, if a sufficient quantity of people feel empathy for Amara and hate the fact that she's not given human rights, then Amara will be given "human rights" (such as the right to own property, the right to self-determine to whatever extent is possible within the boundaries of her personhood and physical limitations, the right to vote, and the right to file suit where her rights are breached). If not, she'll just be thought of as a project.

Probably there will be significant controversy no matter what. And there are some natural problems which will have to be figured out by the society in question (the result will be determined by the society, so I can't really say here what that would look like). For example, if Amara has the right to vote, and if her vote is in any way influenced by her base programming, then what's to stop her programmers from just generating 10 billion copies of Amara and giving them all votes? Such communities of AI will have to be worked into the representative system in their own unique way. Perhaps laws will be made to limit the rate of production/reproduction of new AI in order to prevent the legal system from being overwhelmed by new persons -- or else a technocracy will have to be quickly agreed upon and implemented.

Another example, since she herself is the intellectual property of the company, then in what way is her personhood abstracted from her design so that the company doesn't own "her", but simply owns "her code"? A fine line will have to be drawn there. And if she's performing a function on behalf of the company, then the company will have to start paying her, which means she'll need to be issued all forms of identification necessary to get a bank account -- full integration into the economic/tax structure of regular society.

Improve this answer
$\endgroup$
  • 2
    $\begingroup$ Another good one ! I really like your Copy example. In the story the explaination as to why they dont just make 1000 of her is that on the one side, most Networks that get Generated are just crazy aka "non functioning" and also, the Computer itself isnt exactly easy to build. Special CPU´s and stuff like that. I do however have another question. As Amara is self Learning and Improving, there is pretty much non of the original code left in her. So could the Company claim anything ? $\endgroup$ – Erik Hall Aug 27 '20 at 20:36
  • 1
    $\begingroup$ @ErikHall Well, most Neural Networks (as we now know them) start with a basic learning algorithm which adapts itself with experience -- they keep adding building blocks to themself, but those are for the most part prebuilt. The way neural networks surprise people is because most algorithms include a small "random" element to their mutation algorithm, enabling them to randomly change the prebuilt building blocks as they go. So, if that were the model, then probably after a protracted legal battle, the Company would end up only owning the original code, but not the final product, like you said. $\endgroup$ – boxcartenant Aug 27 '20 at 21:00
  • $\begingroup$ I love the AI voting block principle. Every right is abused by someone. The voting AI scenario is one I brought up in my answer to this question worldbuilding.stackexchange.com/questions/183751/… $\endgroup$ – DWKraus Aug 27 '20 at 23:29
  • 1
    $\begingroup$ It is worth considering the worst thing Amara could do for future AI is sue for independence, it basically guarantees no company will ever bother making another AI on purpose. $\endgroup$ – John Aug 28 '20 at 1:20

Not the answer you're looking for? Browse other questions tagged or ask your own question.