Movies like The Matrix and video games like The Evil Within have popularized the idea of a simulated reality. I'm talking about a human connected to a computer and able to experience similar things like a person would be able to experience on Earth. The simulation doesn't necessarily have to simulate atoms and fundamental particles, rather like we see on video games structures of landscape and "reality" inside it (which of course would be limited to the size of computing power).

Also, I'm not talking about the entire Universe rather the interior of a city. What server-side physical infrastructure would be needed to achieve this?

  • $\begingroup$ Hi michael! I think we'll need a bit more information to help you answer this question, as right now it's very broad. Are you asking about the server-side physical infrastructure required to run such a simulation? The algorithms required to render the highest resolution a human eye can distinguish? Or the Kardashev-level society required to make such an undertaking feasible? I've voted to place your question on hold so you don't get swamped with unhelpful answers. You may also find the Sandbox useful! $\endgroup$ – Dubukay Sep 9 '19 at 5:12
  • $\begingroup$ Thanks for help! $\endgroup$ – michael smith Sep 9 '19 at 5:20
  • 1
    $\begingroup$ I made a few additional edits to help with grammar - feel free to modify anything more. Your edits have been super helpful figuring out what you're looking for, but you've introduced two separate questions here that would each be best as a stand-alone (physical infrastructure vs brain-body connection). I'd recommend leaving off the "breakthroughs in neuroscience" and stick to the physical infrastructure required to simulate a city $\endgroup$ – Dubukay Sep 9 '19 at 5:34
  • $\begingroup$ So you want me to delete part about neuroscience? $\endgroup$ – michael smith Sep 9 '19 at 5:50
  • 1
    $\begingroup$ The physical infrastructure size is a bit on the side of overly broad. The part about neuroscience definitely makes this question very broad. I would recommend limiting yourself to one question about one topic instead of trying to cover everything. Also what sort of city are we talking about? How many people live and move around? $\endgroup$ – Shadowzee Sep 9 '19 at 5:52

Simulating the physics is a tricky concept. If you were born in the matrix, how would you know any physics was "wrong?" Only those who escaped the matrix would know.

However, we can do some herustics on scene generation. Our movies are the closest to "photo realistic" we're going to get, so we can use that as a benchmark. Transformers: Dark of the Moon provided an interview in which they said the most complex scenes they rendered took 288 CPU hours per frame. Multiplying that by a crude 30 frames per second, and you need just shy of 900 CPUs to keep up. With, say, 10,000 residents, that's 9,000,000 CPUs, dedicated to nothing but rendering.

What does that look like? We don't make them that big. To give you a sense, here's Mira Mira

Mira is a Blue Gene/Q hosted at Argonne National Lab. She's got 65,000 processors.

So imagine Mira, cloned about 150 times, side by side. That's your scene generator.

Now, that being said, the problem is actually worse. Those speeds assume rendering is perfectly parallelizable. It isn't. So you'll run into issues there. You could also save some time by only rendering what's in the fovea of each person, but now you have to be able to respond to changes in the eye within the blanking period of a saccade. That's a hard real time challenge. But at least it gives you a sense of scale.

  • $\begingroup$ I'm confused by your maths here. If you need 288 CPU hours per frame, then you'll need 8,640 CPU hours per 1 second (at 30fps). But that's 8,640 CPUs running for an hour to generate 1 second of rendering. So this would mean you'd need over 31 million CPUs for real time rendering per person. $\endgroup$ – Kyyshak Sep 10 '19 at 15:22
  • $\begingroup$ @Kyyshak You're right. I dropped the hours-to-seconds conversion $\endgroup$ – Cort Ammon Sep 10 '19 at 18:07
  • $\begingroup$ @Chickensarenotcows I just wanted to provide a back-of-the-envelope lower bound estimate. Even just doing the oversimplifications of our input that I did, the result is still an astounding amount of infrastructure. $\endgroup$ – Cort Ammon Sep 10 '19 at 18:07

If you are making virtual reality in Matrix style, where you plug directly into the brain, instead of putting displays in front of your eyes, you can inject the sensations lot more directly. You can save lot of computing power.

If you are playing a videogame today, the computer has to take the models, textures, etc and render them into a photo-realistic picture that your eyes can see, then your brain breaks it back down, recognizes the parts and creates a mental model for you. You could skip the whole rendering part and just inject the model into the brain.

Two examples from everyday world:

  • Dreams. When you dream, you can "see" all kinds of things, but your brain does not create a precise image that would be fed back into the nerves that come from eyes, the dreams are created deeper inside as more abstract concepts.
  • Drawing what you imagine. All of us can imagine almost anything that is described to us - "a small cottage at the side of a river in front of beautiful mountains" - you can imagine it and "see" it clearly in your head, but unless you are trained artist, it is difficult to draw this to paper. It is partly because what you "see" in your head is not a 2D image, it is a mental model created by your brain as-if you saw it.

Today we have very little knowledge how is the mental model structured in the brain and how much it differs from person to person, but in the future we may know more and we may figure out how to affect that parts of brain directly. Then you would change the computer simulation to represent the world in a way that would be optimal for turning into mental models that are injected into people's brains, which is likely quite different from the representation that we use today for rendering.

Interesting outcome of this is that if your simulation is not good enough, people won't see pixelated textures or low-polly models. It will rather feel like a fuzzy dream. You see something, but you can't get yourself to focus on details.

A quote from Matrix the movie:

Neo: Do you always look at it encoded?

Cypher: Well you have to. The image translators work for the construct program. But there's way too much information to decode the Matrix. You get used to it. I...I don't even see the code. All I see is blonde, brunette, red-head. Hey, you a... want a drink?

The Matrix code is some kind of a transcript of the mental model that would be sent to person's brain in that area.

So at the end we don't know how much computing power will be needed to simulate the world in Matrix-style virtual reality, but it should be lot less than if you try to render the view of every person as an image. I would compare it to servers for massively multiplayer online games today, rather than to rendering farms. A MMO that puts all players into single world, rather than into multiple copies of the same world. Such as Eve Online.

Here is Eve Online's Server Model.

Each of EVE's 5000+ star systems is loaded as a separate process onto any one of hundreds of IBM blade servers, with some high-load systems being given a server all to themselves and many low-load systems being combined and run on servers together.

EVE's server houses over 300,000 players with a peak concurrent user record of over 40,000.

40,000 could be a population of smaller city. Of course simulating real-world-like experience will be harder than simulating game with ships in space.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.