2
$\begingroup$

Let's say I have the ability to create a fully immersive VR experience a la The Matrix. The quality of this simulation is sufficient that a person "inside" cannot necessarily¹ tell the difference between the simulation and reality.

This is also, as you can probably imagine, really expensive, such that only the super-rich can afford it. However, it is much, much cheaper to record what someone experiences in a simulation and "replay" that for someone else, essentially fusing simulation with the signature tech of Total Recall.

Now... here's the rub. Producing these memories "requires" a real human to experience them in a simulation². Either we can't just create the memory directly on a computer, or memories produced in such manner are significantly inferior.

Why is this the case?


(¹ For all sorts of reasons, we might want to intentionally rig the simulation so that the "occupant" knows it's a simulation, but that's not really relevant.)

(² ...or maybe in the real world, but the recording equipment is problematic in that case. It's much easier to record an experience when the person's physical body is already jacked into a stationary machine.)

$\endgroup$
1
  • $\begingroup$ Are you talking about just virtual reality (Matrix-like), or implanted memory (Total Recall-like)? $\endgroup$
    – Alexander
    Jan 15, 2020 at 0:19

4 Answers 4

5
$\begingroup$

It is much cheaper to use customer experiences because the customers are paying you.

You would need a technician and setup to simulate a person wandering around thru your VR. It could be done. But you already have many mucus-filled pods of people who have paid big bucks to actually experience wandering thru your VR. It is free for you to record their experiences - better than free because they pay. Also you have a lot of raw material to work with. Why pay the tech?

Now, you might make a replay sim by cutting and pasting the best parts of customer experiences and getting rid of the boring bits. Frequent users of such sims might recognize good snips that had been used before.

$\endgroup$
2
  • $\begingroup$ You make a good point! I actually hadn't thought about this... although I wonder how many of those high-paying customers are willing to let their simulations be recorded. Of course, the obvious answer is that there is a "privacy premium"... (Unfortunately, this doesn't exactly work for my story, which is about someone whose job is to create recorded experiences.) $\endgroup$
    – Matthew
    Jan 15, 2020 at 15:51
  • 1
    $\begingroup$ I like the idea of the creator creating out of samples. Sort of like 1990s rap music or Beck. I can imagine him coming upon a strange unlabeled file of samples to use. On experiencing them he emerges with "what the hell was that??". $\endgroup$
    – Willk
    Jan 15, 2020 at 16:42
2
$\begingroup$

How the brain operates is really complex

This is literally taken from the real world. Here is a very brief history of AI: right about the 1960s the AI field was picking up pace. Researches were theorising that we'd have fully thinking AIs within a decade. As you can see 60 years later - that's not the case.

OK, just making a computer think was proving to be really hard. But surely if we just fully simulate a brain on the computer, we'd get an AI. It'd be cheating, since it's going to be a human brain, just in a computer, not teaching the machine how to think but - hey, it'd work, right? Well, the only problem is that nobody knows how the brain works. So, we can't simulate it on PC because we literally don't understand it.

And this has been a brief history of AI. We've sort of stayed right there over the past half a century. I don't mean to undermine what AI research has done, since it has advanced a lot but the expectations in the beginning were just too high. And since we still don't know how brains work, it's pretty hard to make a machine really think, since the easiest path is for it to be like us.

So...in your world, that's still the case. Maybe researchers have found a way to "download" or "copy" experience and replay it but making a new one is still impossible.

I'd liken it to animation/video. We don't actually capture on film what we see. We capture a series of still images. So, in the real world you can see a cat doing a funny trick, when you capture it on camera, you essentially snap dozens of pictures every second. When you play the picture in sequence fast enough, you can trick the brain into thinking it's not still pictures any more but actual motion.

Perhaps the virtual experience works sort of similar. You don't take pictures of the brain but perhaps record somebody, say, eating ice cream and record signals that go to the brain. Then somebody comes in and wants to get one virtual ice cream experience - you re-play the recording the correct way, and you can trick their brain into thinking "hey, I'm eating ice cream!". You could have some leeway into what actually happens in the simulation - similar to how people can train themselves to take control of their dreams, perhaps the simulation is similar in that you can induce it on somebody and they can then take spin it off. They can't go from eating ice cream to riding the Ferris wheel (you're still running the ice cream recording) but they can perhaps change the flavour or imagine themselves on the beach.

$\endgroup$
1
  • 1
    $\begingroup$ Actually... on further reflection, I've promoted this from "interesting and useful" to "absolutely viable as a direct solution". I think on initial reading, it didn't sink in, but you definitely have a point about whether or not it's even possible to synthesize "an experience". I'd sort of assumed it would be, given my other axioms, but thinking about it further, it's entirely possible that there is a categorical difference between simulating sight/sound/etc. and simulating memory. (Which is odd, because I was sort of already thinking along those lines...) $\endgroup$
    – Matthew
    Jan 17, 2020 at 17:13
1
$\begingroup$

The primary reason, known by most sci-fi types, would be the Turing Test. The test requires an observer knowledgeable in the field of AI (thus knowing what to look for) to be unable to tell the difference between two participants in a written natural-language conversation, one of whom is known to be an artificial intelligence (but which one is unknown). Numerous variants exist, changing who the observer and human participant in the conversation are and how much any person involved in the test knows about what's happening. It boils down to the fact that a critical milestone in AI is for an AI to be able to appear human to an observer looking for the opposite. No AI has yet passed the Turing test, though we're progressing faster toward it now than we have at any point since Turing hypothesized the test.

The Turing test, in turn, is only the first step. An AI that can pass for human in a written conversation can still fail miserably at doing the same in a spoken conversation, due to a human's ability, honed over millenia of evolution and language development, to recognize and interpret the spoken word. Again, we can get really close in specific cases; text-to-speech algorithms are now ubiquitous in our society with systems like Alexa, Cortana, Siri and Google Assistant, as well as other systems like ASOS/AWOS broadcasts to pilots, National Weather Service reports for the rest of us, and other systems designed to naturally and intelligibly recite a formulaic subset of the English language. However you can still hear the flaws in the algorithm's cadence and pronunciation, especially of words with unintuitive regional pronunciations including proper names.

Those problems have to be solved to a far greater degree than even many movies have shown the tech to have progressed under purely human development (i.e. Jarvis in the MCU, who's smooth, but still a bit measured; the demonstrably more complex intelligence Ultron, and their hybrid Vision, could probably pass the test if they existed in reality), with a percentage of processing power that still allows the simulation of hundreds or even thousands of humans at least at a basic level of behavioral observation and casual conversation, in order for a human to be able to enter this simulation and be unable to tell whether any arbitrary avatar has a human or an AI driving it.

Then, all you need is a level of real-time graphical rendering such that you couldn't tell the difference between an avatar and a video recording of a person at the same resolution as your VR display. Again, we're tantalizingly close: Detroit: Become Human - Voice Actors and Cast. Skin's still a little smoother and shinier than it should be (especially for more aged characters), rendering is still a little "overaliased" to hide other flaws of modeling and lighting, but games like Detroit: Become Human are basically playable movies, drawing from what's now over 30 years of pre-rendering expertise from the actual movie biz, and decades of driven development in real-time graphics processing.

Put it all together, and you might create a totally synthetic environment and series of events which a human would buy into as being real, or at least real enough that people would choose it over actual reality. However we just aren't there yet, especially regarding an AI holding a conversation with a human that would leave that human unable to tell whom - or what - they had been talking to. Because we cannot do that, we cannot - at this time - create a purely synthetic memory with zero human material input. Everything else is literally a question of sufficient processing power to simulate the "look and feel" of reality closely enough and quickly enough that you couldn't tell.

$\endgroup$
7
  • $\begingroup$ Care to comment, downvoter? $\endgroup$
    – KeithS
    Jan 15, 2020 at 15:57
  • $\begingroup$ (Yes, was already writing it...) Unfortunately, this doesn't actually address the question. I've stated as an axiom that I can immerse a human in a live simulation such that they can't tell the difference. Your post is entirely about why that's hard... which is true, but again, I'm taking as an axiom that I've done as much. The question is why there is a difference between recording a simulation "fed" to a human versus synthesizing such a recording. $\endgroup$
    – Matthew
    Jan 15, 2020 at 15:57
  • $\begingroup$ That said, I do notionally have a limit on number of intelligent actors that can be simulated; if you're paying close attention, one possible "tip off" to a simulation vs. reality would be the lack of crowds, or lack of individualized behavior in a crowd. However, most simulations will simply avoid having crowds. $\endgroup$
    – Matthew
    Jan 15, 2020 at 15:58
  • 1
    $\begingroup$ You proposed a VR world that is physically indistinguishable from reality; my answer pointed out that behaviorally, making an AI indistinguishable from human is hard enough that even the Matrix universe Agents wouldn't have stood up to detailed inspection of their humanity (the point being they'd never be under that much scrutiny of their behavior by the human public populating the Matrix). So that was the point; you can make the environment hyper-real, but without really good behavioral intelligence, the simulants can't pass for human even if a still image would pass for a live camera photo. $\endgroup$
    – KeithS
    Jan 15, 2020 at 16:22
  • 2
    $\begingroup$ The question you asked posits a perfect simulation of reality, then asks why it would have to be imperfect in a particular way (requiring humans to create the simulated events). The only possible answer to that is that the AI isn't good enough, because it cannot "imagine". The simulants, and/or an overarching AI directing the simulation, are not capable of conceiving a wholly original situation that is both interesting as a narrative and believable to the human participant. A human therefore has to produce the basic narrative in some way; the AI is there to improvise. $\endgroup$
    – KeithS
    Jan 15, 2020 at 16:55
1
$\begingroup$

First off, AI may or may not be able to "create" a sufficiently interesting simulation in the first place. However, this is trivially solvable by having human "script-writers", which does not directly address the question being asked.

So... why do we need a human to actually experience the simulation?

Well... lots of reasons!

A simulation isn't "perfect" by itself

Here, we can take inspiration from some of KeithS's comments. The question took as axiomatic that the simulation is "indistinguishable from real life", but maybe that is only true when a human is actively in the loop. For now, let's only consider the "external" simulation factors, i.e. stuff that would be an issue if we asked an AI to produce a movie all by itself versus hiring actors. Perhaps an AI on its own is not capable of producing a believable synthesis (even with a human to "write the script"), but can do so when working in real time with a human agent.

This could work using a combination of techniques. First, by being able to "read" the expectations of the participating human, and "second" by being able to manipulate their sense of time, possibly even short term memory, such that the simulation can jump back and "retry" and segment that was problematic.

A simulation doesn't react the way a human does

Part of what we are recording is the way the human participant reacts to the simulation. This includes obvious things like emotion, but also more subtle things like what holds the participant's attention at any given moment. Modern video games already do a reasonable job at simulating external stimuli, but the stuff that goes on inside the head of the human experiencing the simulation may well be a wholly different matter.

A purely synthesized experience might be like watching a movie while on a high dose of mood-depressors; sounds and images are present, but they lack emotional impact. Focus of attention might shift in was that are confusing or even unsettling.

Additionally, an AI hasn't lived a human life. The way in which a human experiences a simulation will be "shaded" by his or her prior life experience, and perhaps the "playback" carries this as well. An AI will surely have difficulty relating the events of a simulation to memories of its earlier life in a way that another human can comprehend.

Again, maybe we can throw more simulation at all of these issues, but we are now trying to solve a problem that is plausibly much harder than simply producing a believable external simulacra. Worse, attempts to simulate these aspects might fall prey to the uncanny valley effect.

Cost aspect

Humans are highly adapted and very efficient at recognizing, and even in some cases, conceiving, "realistic" stimuli. Even if we hypothesize that an AI can create a totally believable simulation, there is still a cost in "processing cycles" to do so. It may well be that it's just cheaper to offload the work of translating an idea for an experience into a recorded experience to a "wetware processor". This becomes doubly true if even the simulation is able to leverage a human brain's ability to "fill in gaps". In this case, simulations would be less like a next generation VR and more like "directed dreaming", which also ties in again to the first point. (But this would need to be balanced against the goal of full-blown simulations being "expensive".)


In summary:

Creating a believable simulation — that is, dealing only with the simulation of external stimuli of a human's senses (sight, sound, etc.) — may require a human "in the loop" on its own. Also simulating the human response to such a simulation, which is a key aspect of a recorded experience, poses additional challenges that may be on a whole new level... and even if it doesn't, it may just be more economical to use humans for that aspect.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .