Let's say my eyes were ripped out in some horrible accident, and were replaced with cameras. Over the years, the cameras get better and better... 4k... 8k... 16k... etc. Is there a limit to how high of a resolution our brains are able to process?
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ You would probably get more "super-vision" out of cameras with extremely fast zoom/focusing, rather than increased resolution. Our brains could probably deal better with that than slamming it with raw data. A zoom capability would probably "feel" like we are seeing everything in greater detail (even though we are just seeing one small spot at a time). $\endgroup$– Jason KOct 28, 2016 at 13:24
-
1$\begingroup$ Just wire it up and the rest is software, right? $\endgroup$– KysOct 28, 2016 at 17:32
-
$\begingroup$ This might give some idea about the subconscious' visual processing limits: newatlas.com/us-army-eeg-brainwave-image-analysis/40309 $\endgroup$– Thom Blair IIIOct 29, 2016 at 2:53
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
8
Yes, there is a limit.
The camera would have to use the same communication protocol that the natural retina uses to feed into the optic nerve.
The optic nerve isn’t just a wire like a USB cable.
Rather, it is a processing pipeline. It is actually part of the brain, just arranged in a long strip so that the work that needs doing is performed in the physical space that needs to be filled.
We receive not pixels but more like a vector description with perception tags.
But maybe we can use higher resolution anyway?
What if you replaced the optic nerve and as much of the visual processing cortex as was needed, too?
The result of the whole process might be, for example, to “see” (meaning to perceive after analysis) that there are two edges next to each other. The original retina could not resolve that there were two different edges so the analysis only finds one thick line.
But the high-resolution prosthetic eye, with updated analysis circuits to match, resolves two close-spaced features and dumps that information (“there are two closely spaced parallel lines at such-an-angle relative to other features”) to the visual cortex.
Fake augmentation
But maybe science isn’t ready to replace these brain structures. The different (better) retina replacement can have its own processing internally, which then is used to generate fake data compatible with the optic nerve input. Maybe you'll “see” the two lines pre-enhanced or the separation exaggerated; with a general smart zoom feature working to fill your visual input with what’s interesting.
The high-res sensor could zoom by taking a subset of the spatial extent rather than subsampling all of it. It could provide a movable fovea; the brain can't handle more data, but you could select which area is the region of interest.
Also, note that the retina has cones and rods randomly arranged, not lined up in a neat 2D grid! This is important because it reduces aliasing. If the machines are neat grids, you might need higher resolution to achieve the same effect. To provide compatibility, the processing in the eye would clump real pixels to emulate randomly-positioned (larger) pixels. And, it could be auto-adaptive to even better prevent artifacts and improve perception of real details.
answered Oct 28, 2016 at 1:54
-
$\begingroup$ I think there is a loophole that could get around this limitation. The brain is plastic and can reassign functions to new purposes. So, for example, you might sacrifice part of your hearing or you sense of touch in order that the parts of your brain that used to process that now process extra visual information. Something similar happened when color vision evolved. It pre-empted parts of the brain that were previously used to process the sense of smell. $\endgroup$ Oct 28, 2016 at 2:58
-
$\begingroup$ The processing cells are rather specific, and already developed. Implanting a device in a mature organism is not the same as evolving a new feature gradually. IAC, the (main mass of the) brain isn’t getting the raw pixels! $\endgroup$– JDługoszOct 28, 2016 at 3:03
-
$\begingroup$ But, there are precedents for adult adaptation, such as people who are born blind and given implants to allow sight. Repurposed neurons might not be as efficient, but can work. To give another extreme example of repurposing, there really are people who lose sight during life who develop echolocation in lieu of that. $\endgroup$ Oct 28, 2016 at 3:20
-
$\begingroup$ Yes there is some plasticity, especially in the deeper pattern recognition hardware, and in “mapping”. That’s downstream from the specialized sensory input features. And remember, the brain is not getting pixels. If you’re attaching a new retina to the optic nerve, what would you plug the new additional pixels into? $\endgroup$– JDługoszOct 28, 2016 at 3:32
-
$\begingroup$
$\endgroup$
1
Human eyes aren't cameras
We simply don't "see" in the same way that cameras do. What we actually see in any instant is a very small amount. Our brains do an incredible job of filling in the gaps of our instantaneous perception and giving us a vision of the whole.
This is why we're so prone to optical illusions and explains why they highlight how our perception works.
This "gap filling" feature of the brain allows us a huge amount of capability that fools us into thinking we have super-cameras in our heads
We are able to: * Have an almost infinite depth of field * Instantaneously "see" a scene with a huge range of tonality (say you're in a dark church looking out of the door into blazing sunshine) * Have a huge range of sensitivity - we're able to sit in an almost dark room and still see enough not to walk into tables.
If we were to replace our eyes with high definition cameras, our brains would probably not be able to process the added information. We'd have to completely relearn how to see.
$\begingroup$
$\endgroup$
4
Bandwidth limits abound. The human optic nerve is estimated to be capable of carrying about 8.75Mb/s, give or take. Our "flicker rate" of about 30fps would give us about 291kb/f for each image. Using 8 bits per channel, that means we have 36500 pixels to work with. That's a far cry from the millions of pixels in a modern camera.
The human brain and optical channels are brilliantly designed to maximize value out of the whole system. They're not designed for raw data input, they're designed for finesse.
We could replace the optic nerve with wires to get more bandwidth. Obviously the brain has more bandwidth to play with than the optic nerve does. However, it's not clear if we could really leverage it. We already spend 20% of our total metabolism on our brain. Doing calculations costs energy. While the calculations on a full camera image are simpler than they are on our organic images, they'd still rapidly swamp the brain. They're not the kind of decisions the brain is good at.
Interestingly this energy cost is so pronounced that we actually divide our retinal gangelons into two classes. A small fraction are designed to catch rapid motion, and are updated at a high rate, while the majority are "sluggish," designed to update slowly and catch the whole scene.
answered Oct 28, 2016 at 3:44
-
$\begingroup$ But the optic nerve doesn’t cary pixels (like an HDMI cable) at all. It’s a processing pipeline that happens to be physically stretched out in space. IT doesn’t cary data in the manner you suggest; it take input at one end and delivers results at the other, with each hop along the chain performing a step of computation. $\endgroup$– JDługoszOct 28, 2016 at 7:19
-
$\begingroup$ @JDługosz Do you have any resources which suggest what kind of processing is done inside the optic nerve? It makes sense that the body would choose to do that, but I haven't found any links which suggests it happens. I love finding out that my nerves are smarter than me! $\endgroup$ Oct 28, 2016 at 16:31
-
1$\begingroup$ While i know bit rates don't match what the brain uses, if we start talking about cameras, they only talk in bitrates, so it makes sense to use them even if we have to be creative with how we measure the brain. The moral of the story doens't change: use neurons for what neurons are good for, not for what they're bad at. $\endgroup$ Oct 28, 2016 at 16:33
-
$\begingroup$ they only talk in bitrates - because of the results we need from them. If we like to couple them with a brain, this position of bitrate results/approach have to be changed for successful brain coupling. Between the camera which talks in bitrates and brains it have to be a decoder/encoder device which supply the brain with information it may process, in a way it used to get that information. If you do not mean replacing visual cortex, and as result have bigger problem and need in that encoder device. In both cases visual path bandwidth is irrelevant. Or you should say why it is not irrelevant $\endgroup$– MolbOrgOct 29, 2016 at 1:36
$\begingroup$
$\endgroup$
From Wikipedia we learn that the visual field for one eye is (from the centerpoint) about 60° up, 60° towards the nose, 75° down, and 110° away to the side and for two eyes it is 135° vertical and 200° horizontal. We also learn that 1 arc minute is a reasonable resolution, for the purposes of calculation, why not make that 0.1. I'll leave it to you do do the math(Ed note: about 97 MPixel, close to eye physical limits ).
Of course, this ignores response rate, the ability of the lens to focus at different distances, the ability of the iris to increase the amount of (intensity of) the light hitting the retina, the sensitivity of the retina and the color differentiation of the cones & rods. I've forgotten the number, but it was about 3 or 4. The brain SENDS 3 or 4 times as much data TO the eye as the eye sends back to the brain.
In other words, vision is a very computationally dense process. I've seen two things in my lifetime which have amazed me and which are relevant here, I think. I knew a guy who could read a traffic speed limit sign at a distance where the sign (rectangular white) was little more than a dot to me. He had incredible resolution. I knew another guy, (I've noted it in several) who could (and did) "take in" an entire scene in a single glance. Meaning, for instance, he and I would be walking by a room where a meeting was going on, and we'd both glance in. I'd recognize a couple of people and a few other details while he could tell me who was there and what exactly the powerpoint slide of an excel worksheet being shown had on it. (not everything, but an amazing amount of details). In the first case, the resolution is physical (most likely); in the second case he had to have both excellent eyes, and a neural system which was able to process all of that information a lot more efficiently than I can.
The take-away from all of this is that there are all sorts of limits on human vision. Some of this is due to the physics of the eye, some of it is due to the processing in the optic nerve, and some of it is, no doubt due to the efficiency of (effectiveness of) the visual volume/area of the brain. One thing to keep in mind is that the processing power of the brain is limited. The more information you extract from one "instant" of stimuli, the fewer "instants" you'll be able to process (everything else being equal). This implies the perfect system would be one which only focuses on the stuff you want to retain. The downside of this is that while focusing on the strawberries, you miss the leopard in the tree.
