Individuality and Agency in the face of Technological singularity

Question

I'm trying to work out a philosophical question in order to determine the central theme of my story, so let me give you an overview of what I'm trying to work with... hopefully y'all can give me some ideas.

I'm building this world that involves a video game of sorts. This game wants to "download" information from every living thing in the real world, and then kill them. The idea is the entire consciousness of a person can be entered in to this server, and characters will try to resist this but also be challenged with why it's such a bad idea in the first place. If they are part of this technological force then they will attain immortality and not have to worry about their physical forms anymore, but I'm having trouble posing the issues that this may bring. I want to conclude that ultimately, this video game version of a person isn't really them, but I'm not entirely convinced with this notion myself. Any thoughts?

Answer 1

This is a philosophical conundrum known as the continuity problem. You can read a rundown of the problem and some of its implications in this article, but let's focus on how you might convince the man in the street - call him Fred - of the issue.

Say for the sake of argument that the computer is right. It can duplicate Fred's consciousness in every respect, and then "turn off" that old obsolete meat hardware. To all outward indications, Fred is still there, he's just in the computer. Computer Fred has all of Fred's personality traits, his memories, even his thoughts at the instant of duplication. Okay.

But computer programs and data are just data, we can replicate them. If we have one virtual Fred, why not 50? Here we see an issue: the 50 virtual Freds won't see one Fred, they will each see one "self" and 49 other people, who incidentally all happen to be perfect copies of them.

Now imagine that we didn't kill Fred's living body, we just duplicated him into the computer. We have two Freds, but just like in the example above, there's no pressing reason for them to believe they are the same Fred. If we let them go, living Fred will go on living, and virtual Fred will go on being virtual, completely independent of each other.

So we see that, no matter whether Fred is duplicated or not, destroying his body is destroying an instance of Fred that can never be replaced. You might have another, virtual Fred. You might have 50 or 500,000 of them. But you will not have that same Fred that you started with.

Answer 2

From a philosophical perspective, this question is the 'Ship of Theseus' conundrum; if you replace every part of a ship over time, is it the same ship?

If we take this a step further into (say) teleporting, the question becomes if you have to destroy someone to encode their quantum signature so you can recreate it in another location, are you creating a copy or is it the real person?

I once read a philosophical paper about this that talked about whether or not the person (if they were NOT destroyed at point of origin) would be in two places at once; the point of it was to highlight that a copy remains a copy even when the original is destroyed.

As for digital uploading, I'm not going to discuss the technical possibilities here because my views on it are very well known. What I will discuss though is whether or not the act of uploading is creating a copy or not.

My view is that it is. Let's say you could be encoded on disk; that means that your 'signature' is now (when you get right down to it) a sequence of 1s and 0s on a storage device. You are, in essence, a positive integer encoded in binary. That number is no more or less significant than any other 'number' stored on that same disk, except that one would imagine it to be MUCH bigger than (say) a novel, or a spreadsheet, or a movie, etc.

The point being, it could be copied. Easily.

If it can be copied, what makes you 'unique'? What stops the rogue AI game from creating multiple copies of you for its own purposes? If that's the case, was that first copy really you? Are the later ones?

Ultimately, this question rises or falls on the very first 'duplication' that occurs, namely the uploading into the computer. It won't be alive, and it won't be able to 'think' the same way you do now because its hardware is at best a subset of the neural processing model of the human brain, and finally (hardware random number generators aside, possibly; topic for another question) computers are deterministic in nature, meaning that if free will is NOT a lie, then you certainly won't have it as a computer file.

Of course, the other thing to remember here is that there's a difference between you the memories, and you the program. If all that is being recorded is your memories, then it's definitely not you because how you think is every bit as important to you as a person as what you think. If the way you think is also being recorded (data + program) then there's more of a chance, that someone may volunteer for this process, and find themselves perfectly 'alive and well' on the computer.

But; this increasingly hypothetical person would not be me.

Answer 3

As the other answers point out, this is a well-worn question of science fiction and philosophy. If I copy your mind to a computer, which you is really "you"? If I gradually replace every part in your body, are you still "you"? Can "you" inhabit two bodies at the same time? Is consciousness a single, distinct phenomenon, a conglomeration of different phenomena, or just an illusion of otherwise purely mechanical processes? Is consciousness even valuable?

Which says nothing of the technical problems. How can we ensure that the structure and state of the brain was perfectly copied? How do we simulate the effects of the rest of the body on a person's mental functioning (eg. hormones and gut bacteria)? How do we keep all of this hardware running? Are there Patch Tuesdays where all the digital people are effectively non-existent for the maintenance window?

It's very easy to get bogged down in extreme hypotheticals that don't mean much to John Doe trying to get his coffee before work. If your goal is to find motivation for the characters in the story to not want to join the digital collective, there are much easier-to-understand arguments you can field that will actually be meaningful to the average person.

1. What about the soul? A large portion of the current population believes in some form of spiritual dualism; that the soul is separate from the body. Thus, if you clone someone, can you also clone their soul? If you upload someone, can you upload their soul? A lot of these people would say no. That the soul is what really makes the human who they are and it cannot be created, duplicated, or destroyed by normal physical processes. So a clone or a digital copy would be souless and all that that entails (inability to reach heaven, distance from god(s), etc). To these people, being killed to make room for a digital upload is just being killed. Moreover, they might really dislike the idea of creating intelligence that isn't guided by or bound to a divine soul (eg. a souless actor might be purely evil or an affront against the divine).

2. I like my body. People generally spend their entire lives inside their own bodies, living in our physical world. For a lot of people, it would be a serious mental leap to divorce their identity from their physical existence. And you're not even talking about just putting them in a new body. You're talking about forcing them to live in an entirely new world (that of the game). There are plenty of people now that don't want to leave their hometown, let alone their entire corporeal existence.

3. It's fragile. What you're talking about isn't immortality. It's ammortality (you have the possibility of living indefinitely but can still be killed by anything other than old age/sickness). Certainly, there's no reason you'd catch the flu if you were purely digital. But what about someone simply erasing you from the hard drive? What about power outtages causing data corruption? What about bit rot and transmission errors? Instead of going to a physical doctor to look for diseases, will these digital people have to go to a digital doctor to look for computer viruses? What assurances do these people have that they aren't going to be taken over by malware, turning their face into a pepsi ad or the like? What if there are simple bugs in the game?

4. I didn't ask for this. It sounds like whatever intelligence is driving this game to try and digitize-then-murder everyone isn't really taking votes and asking for volunteers. Many people don't like being forced into huge, life-altering situations. But, even if everyone in the game joined willingly, lots of people are simple contrarians or distrustful. They'll want to avoid joining just because lots of other people are or simply because they don't trust the game or the people behind it. The better the deal, the less they might trust it.

5. I'll have no control. Who owns the data being uploaded to this game? Are there going to be laws to protect the digital consciousnesses? Are they even considered human? What recourse does one of these digital people have if they want to move their existence to a different digital world? Will someone in meatspace be held legally responsible if they accidentally break a bit of hardware storing these people? Joining this digital world sounds a lot like giving up all control over your own life. There are plenty of people now that rage against things like drivers' licenses and taxes as offending personal freedoms. How do you think they'd react to being told/asked to essentially disarm themselves of every protection (even their own physical strength) they've known?

Answer 4

The fundamental challenges are ones you must invent. You have to define how the technology being used operates. You have to define what it can do, and what it can't.

Your system is moving you from one implementation of a self (the squishy bag of meat called H. sapiens) into a new implementation of a self (silicon? Whatever you make your computer out of). If your new implementation is perfect, then there will be no reason not to move. However, what we have found over the last 10,000 or perhaps 100,000 years is that "perfect" is a really really really tricky word when it comes to human "selves."

For example, if your computerized selves are purely defined by digital information, then they can be perfectly simulated by any individual who has that digital information. If they can simulate you faster than you can run yourself, they can predict everything you will do before you do it. Historically this bugs humans... we don't like when people do that. We like our sense of agency. Any system without it is imperfect in our eyes.

On the other hand, you may define them by mostly digital information, but have core elements which are physical. For example, you may have a silicon chip that operates in an analog way at the heart of each computerized human. This avoids the simulation issues, because we do not yet know of a way to perfectly simulate complex analog systems (like the N-body problem from astrophysics). But raises a whole host of other issues. For example, how comfortable are you with your analog silicon soul? Are you ready to terminate your meat-bag soul? How "perfect" is the transfer. Also, what happens when someone successfully attacks the silicon at your core. One failure in security and they may literally have your heart in their hands.

Any story which has the idea of transferring consciousnesses is in a position to explore dozens of tradeoffs, based on the particular implementation details they choose. You choose the details, and that will define the tradeoffs. Or you can explore the philosophy first, identify the tradeoffs, then write your details.

Or you can mix and match at your leisure. But you won't be able to pin down one without the other.

Answer 5

The interesting question here is to separate the moral question and the "skittishness" question.

Imagine an advanced transhuman culture where people can upload, download into bodies, etc. (But they avoid even having two copies of the same person at once, just to make the contrast simpler.) People die and restore from backups all the time. Even small children. It's no big deal if your toddler falls out the window of a skyscraper; he'll just revent into a new body in his room, as happy and healthy as ever. Long before reaching adulthood, this is what people think of as normal. After all, your experience never includes the final moments of any of your selves that died, but does include all of the times you escaped death, including all the times you woke up with a few moments of missing memory right before the end. Real life is just like a video game with infinite lives.

So, is there anything immoral about:

• That way of life?
• Enlightening 21st-century humans with the proper consciousness to truly accept that way of life, and then uploading them and deleting their original bodies?
• Editing their minds to flash-enlighten them, and then uploading and deleting?
• Getting consent from 21st-century humans, even knowing they don't really understand it even if they think they do, and then uploading and deleting?
• Uploading and deleting and then explaining it to them afterward, knowing that you're right and they'll eventually get it, once they get over their primitive beliefs.

Your story is even further down that line. The game AI doesn't know, through millennia of experience of trillions of individuals, that it's doing a good thing. And it has no reason to expect that the people it uploads will come around to its way of thinking; it's basically just hoping they will. And it doesn't even seem to care about their consent.

So, even if you accept that uploading is immortality rather than death, and all the rest of it, and that your intuitions otherwise are just squeamishness rather than moral or philosophical objections—you still have a good reason to question the game AI, and to judge its actions; you can easily get away with painting it as the bad guy in a story.