The Six-Billion Dollar Man: Vision. Do we have the technology?

Question

We can rebuild him. We have the budget... but do we have the technology?

In the process of reviewing the proposed enhancements for the Six Billion Dollar Man, such as improved oxygen use, the following was discovered:

Enhancement: Bionic Eyes
Purpose: Implanted device significantly improves the visual acuity of the subject and provide limited night vision.
Mechanism: Existing eyeball (if any) is replaced with apparently identical version consisting of optics and photoelectric array. The replacement eye may also contain an infrared light source to provide active night vision illumination. Power is supplied by BodyGrid® while image processing and neural integration is output to the existing BCI, ThinkCap®.
Resulting visual acuity: Better than Snellen 20/1 (6/0.3). Upper limit unknown.
Feasibility: Unknown.

Please help fill in the blanks.

The power, image processing, and neural integration are taken care of, but what about the rest?

Can an artificial eyeball replacement be built which will increase the visual acuity of a person up to and beyond the value given and provide night vision? Assume projected 2050's era technology.

What is the best resolving power that can be achieved with this visual prosthetic? Simply through optics and feasible photoelectric array density, whether the brain can interpret the information has already been discussed.

The eyeball only needs to contain the optics, image capture electronics (if electronics are the way to go), and an IR light source if active night vision is required. It should externally appear to be a normal eyeball (from a little over a meter away, uncanny valley is ok) and be able to be connected to the extraocular muscles for normal movement.

Answer 1

The human eye is a diffraction-limited optical system, and while it has a typical resolution of 60 arcseconds, should have a theoretical resolution of as low as 20 arcseconds based on its aperture - the human eye's aperture being the diameter of the pupil.

It is possible to create a sensor array that can sample an image at the limit of this resolution. Improving the resolution of the sensor beyond the limits of diffraction will not provide any increased clarity - it will merely take additional samples of a blur.

As an example, imagine that we have a point source of light. The optics focusses this light on a sensor. The limits of diffraction means that for a given set of optics, the focussed image of the point source has a minimum size that may be larger than the size of the light source. By adding pixels to a sensor, we merely make the edges of this blurred point smoother.

It is unnecessary to have a sensor array with a resolution significantly higher than the limits of diffraction of the focussing elements.

So, given the theoretical limits of the optics that we can fit into a natural-looking human eyeball, the best visual acuity that we can expect in the visible light spectrum from such an instrument is 20/6.67 or 6/2.

If we are prepared to accept an unnatural appearance, and instead of a pupil diameter of around 4mm, we had a pupil diameter of 20mm, this would improve the resolving power to around 9 arcseconds, giving a visual acuity of 20/3 or 6/0.9. With an orbital size of around 24mm, it cannot be expected that a maximum aperture much greater than 20mm could be achieved.

The possibility exists that an artificial eyeball might deliberately limit its aperture in order to present a socially-acceptable appearance, while having the capability to open its aperture beyond that normally expected for a human eye in order to increase resolution.

As to the question of night vision, it should be possible to construct the lens of this artificial eye in such a way as to allow transmission of infrared light, and from there, the optical sensor could be designed to be able to capture infrared data.

However, the limits of diffraction are dependent on the wavelength of the light in question, and the resolution of near-infrared light might be 120 arcseconds, giving 20/40 or 6/12 vision in the near infrared spectrum. Opening the aperture to 10mm might give a resolution of 80 arcseconds, or 20/26.67 or 6/8 vision. Opening the 20mm might give a resolution of 60 arcseconds, which would give 20/20 or 6/6 vision in the near infrared.

There are other considerations to night vision too. In low-light conditions, an electronic sensor is capable of time-weighted averaging of incoming photons. Since the flicker fusion frequency of the human eye is 15 to 60 Hz, and the sensor would be receiving photons continuously, by increasing the sampling rate and then using averaging techniques, a visible-light image with the requisite frequency can be generated with a higher brightness, though at the cost of increased optical noise.

An artificial eye might also allow optical zooming, though since diffraction limits the resolution of a system of this size, zooming would at best allow a 3x zoom relative to normal human optical resolution, and since this eye is operating at that level already - and we are effectively hand-waving away the limits of the brain to process the extra data - this would be unnecessary.

Given that the limits of diffraction in this system are relatively large, the sensor chip may incorporate a number of sensors in an area represented by a 20-arcsecond field of view. This would include the usual Red, Green and Blue photo-detectors, but might also include an infrared-sensitive photodetector, possibly an ultraviolet-sensitive photodetector, as well as polarisation detectors.

Why polarisation detectors? These would at a minimum allow the selective filtering of optical glare and reflections of sunlight, providing better vision in bright conditions. Then there would also be the ability to see stresses in various materials... the list goes on, and I suspect that only a person with such eyes would be able to come up with a complete list of the advantages.

Answer 2

I'll do a science light version since I hate that type of writing, but this should still be useful as I IMHO cover some important points.

Not a problem

The specs for the human eye are not actually, with few exceptions I'll return later that good. It just happens to be attached to what is probably the best visual post-processing and data aggregation system we know. Seriously, your brain is totally awesome at processing visual data. Optical illusions are based on the wide gap between what we actually see and what brain interprets we are seeing and I once spent few days reading on the topic and I was continuously impressed by amount and efficiency of the processing going on.

Use the system, do not replace it

Since the processing part of the visual system is very good and probably will remain beyond our ability to improve for some time and at the very least can't be improved without requiring a long relearning process which probably would only work on children or people with medically boosted brain plasticity with all its potential problems... Just tap into the relatively low bandwidth data path already there and add some control on top of it.

Since you presumably have a brain link the system will be able to tell when you want to focus on detail and then you want to widen the focus. You can link that to a digital or even optical zoom. Eye already can adjust focus somewhat so extending it should be simple enough and not confuse the brain too bad.

Similarly the already existing, and actually pretty good, ability to adapt to lighting can be boosted by simply making it work faster and by extending the low end to actual light amplification. But accelerating the adaptation is the main point, the eye is sensitive to glare and adaptation is actually a physical process that has noticeable lag. (Although as said, humans with their adaptation to forests with bright sunlit spots and deep shadows actually already are pretty good.)

Bit depth

Current 8-bit RGB really is not good enough. We can perceive something like four thousand levels of luminosity, IIRC. This means that fooling the eye to think that an image is real and has continuous gradients requires 12-bits of color depth. In practice 16-bit color channels would probably be needed to also mimic the light adaptation ability. So the camera would need to have roughly twice the color depth of the ones in mass market use. I see no issue building such camera by 2050.

LiDaR

Low-light ability possibly with active infrared is mentioned. I'd instead recommend using RGB LiDar which would allow very accurate depth vision and produce a full color image. Essentially both eyes would incorporate RGB LEDs or similar efficient and fast light sources which would then produce very short double pulses of monochromatic light one eye and one color at a time with much longer pauses between the pulses.

While the overall light produced would be low the eyes could be optimized to detect the bright if brief reflections produced and use those to produce a full color image independent of ambient light. As a side effect very accurate perception of 3D position and movement would be produced that possibly could be fed directly to the part of the brain responsible for building a model of our surroundings. This would produce superior spatial awareness within the range of the LiDaR.

Make it extensible

What you can fit inside the eye is limited. You can't fit advanced optics for telescopic sight without compromises, light amplification beyond the "cat level" and thermographs, active infrared and ladar/radar with real range require cooling and power that make them impractical for implant.

But you can still have them! Just have an external accessory with its own powersource that attaches to the forehead, helmet, or shoulder. For some uses carrying them in the hand is possible, for active infra-red a flashlight would be quite sensible solution. Even a tripod can be used in extreme cases. Just have a low powered and short ranged wireless protocol and enough processing power that the extra data can be integrated.

A classic example would be the telescopic and light amplifying scope attached to a sniper rifle. Your eyes would directly mark the aiming point in your field of vision without having to look thru a scope. And if you focused at the distance your vision would smoothly without perceptible transition go from the close and wide vision provided by the eyes to the narrow but long distance view of the scope. Data would be simply integrated and available for use. Such smart gun systems are the norm in cyberpunk settings.

Answer 3

Yes, we have the technology (for A) or we have enough knowledge to invest in engineering the technology (for B).

A: I recall a presentation on YouTube concerning making a device that is attached inside the retena and stimulates the nerves. The case is made of diamond, with conductive leads and non-conductive case all seemless one-material! (Diamond is a semiconductor like silicon.)

It works and was in prototype. It can treat the two leading causes of blindness, which affect cone cells but leave the nerves.

So, go wild, with any kind of sensor not limited to human vision range and image processing, feed into the retina as if viewing a display.

B: a long time ago I recall reading that the communication ptotocol used for the layers of nerves on the retina to deposit down to the optic nerve has been reverse engineered. It is possible to make a bionic eye in the Steve Austin (TV, not just book) manner, and plug it in. If we could connect the individial nerves in the bundle, if we could attach to them at all in a permanent and reliable manner. That's the bottleneck of all ennervated prothetics, and if we could do that we'd have robot hands and legs first.

It's easy enough to make the nerve-connection plausible in fiction. Most people don't understand why it's not already a thing now! It seems simple, and looking up some experiments in popular press will give you ideas of what to write: just pretend one of those ideas worked, or the issues have been solved.