# How much processing power would a real time Edited Reality headset need to render video in real time

Imagine a VR headset with built in camera & AI with object recognition & photo editing software.

The camera records the image.

The AI is trained to pick out 'undesirable' images, objects, shapes & faces.

Which it edits out & replaces with the correct / extrapolated background from earlier footage.

The same is of course done for any undesirable sounds (or voices) with active noise control headphones built into the headset.

Rendering these doctored images in real time so the wearer doesn't experience any noticeable or significant (or dangerous) delay between events & seeing them may take a lot of processing power.

So the question, how much processing power is needed (for hi definition video quality in real time)?

Oh! & (assuming current technology) how heavy will the chips be that we need?

• Can you define "undesirable"? Nov 6, 2019 at 19:57
• @Ryan_L : Nope, the user defines 'undesirable', it might be a specific person, all people, cats, dogs, billboards advertising tampons, what's 'undesirable' is entirely within the gift of the owner of this device. Nov 6, 2019 at 19:59
• And I assume the user also defines what to replace the undesirable images with? Nov 6, 2019 at 20:01
• @overlord : Generalised, if I wanted an exact figure I'd have tagged it hard-science rather than science :) Nov 6, 2019 at 20:04
• @Pelinore Not all people know which tags are appropriate, I was just double-checking. Nov 6, 2019 at 20:06

If we take as baseline current PC reqs, we have CPU I5 + 500/600 USD video card.

So let's forget about the CPU focus on the GPU for a while. Let's take as reference Nvidia Gforce RTX 2080. They got a different architecture than a CPU because it is specialized hardware. 13.6 billion transistiors, 2944 processing cores. The boosted clock gives 1.700 MHz with a bandwith of 448 GB per second.

Without going into the CPU you already got an issue of power supply (215 watts) and proper cooling.

Card alone stands around 700 grams, no packaging.

Now the I5 CPU weights about 200 grams. Let's say it got 4 cores, 8 threads and about 4.20 GHz. maybe more, maybe less.

So summing up, bare minimum a kilogram of hardware with current tech. The specs can be looked up on gaming websites but the previous info gives you a solid baseline.

• Indeed. The main issue with current hardware is size. For a PC with access to 220v that is a non issue. For a wereable device, it will run hot and be uncomfortable. Unless you carry a backpack with all the doodas and stream the images to the headset. Nov 6, 2019 at 21:13
• WiFi with the bulk of the processing done elsewhere then perhaps? Nov 6, 2019 at 21:21
• You can also probably wear the components somehow attached to your clothes. Chainmail is heavy when in a bundle but apparently not THAT noticeable when you wear it and the weight is distributed on your body. So if you apply the same, you could have a "vest" of processing units to wear. It would probably keep you quite warm, though, so make sure you only put it on in the winter.
– VLAZ
Nov 6, 2019 at 21:42
• @Gustavo They also hate it and it ruins their knees and backs. Not the kind of thing you'll convince many consumers to use. Nov 6, 2019 at 22:01
• So essentially get a decent gaming laptop in your backpack plus a helmet the size and weight of a motor cycle helmet. Nov 7, 2019 at 10:06

In my eye, the limiting factor might be RAM. The human eye has a resolution of about 120 megapixels. Given 32bit color, meaning each pixel takes 32bits to store, we're looking at needing ~56 gigabytes of RAM just to store one second worth of video. You can't compress it, because you need the raw video to do your filtering. This isn't counting whatever else you have to store to do the calculations, like your archive of undesirable images. It's also not counting anything to do with audio. I don't think it's far-fetched to say we're looking at a terabyte of RAM. The first RAM chip I could find on Amazon is 16gb, costs $77, and weighs 3.2 ounces. So using those, you're looking at almost \$5000 and 13 pounds just in RAM.

• The human eye does not have a resolution of about 120 megapixels, not even close. You are wrong by an order of magnitude. Simple experiment: go to a shop selling fancy TV sets. Find a large 4K TV showing an image; that's 8 megapixels. Approach the the TV set until it covers a field of view of about 90° (outside this field of view the discriminating power of human eyes is very low and can be safely disregarded). Can you see individual pixels? Nov 7, 2019 at 0:34
• @AlexP I just googled "resolution of human eye" and used this article from Forbes: forbes.com/sites/quora/2016/10/06/… Nov 7, 2019 at 1:38
• Forbes, the well-known source for anatomical and physiological information. In brief, the acuity of the human eye is not constant over the entire field of view. There is a central area of maximum acuity, where the eye can distinguish points at about 0.5 minute of arc separation; say that this covers 10°, and let's bump the acuity to 0.25 minutes of arc. This gives about 12 megapixels equivalent resolution for the central area. The rest, up to 90° is much coarser, but let's be generous and add 12 megapixels more. Total is about 24 megapixels and that's with very favourable approximations. Nov 7, 2019 at 1:48
• If you prefer hard anatomical data, a foveal cone, that is a cone in the region of maximum acuity, subtends about 15 arc seconds... Nov 7, 2019 at 1:52
• Your estimate of 3.2 ounces or roughly 80 grams for 16gb is nowhere close to optimal either. You can get ram chips of 64 gb which immediately cuts your estimate by a factor of 4. 80 grams also seems high and might include some packaging. Nov 7, 2019 at 10:03