Brain-construct

Question

I am currently reading an eternal golden braid, i've gotten as far as several lengthy chapters about the brain.

He talks about neurons and symbols, what 'meaning' is and there is a lot of hypothesizing going on. To me much seem like assumptions about our mind from a mind strongly affected by its knowledge about computers. Anyway it intrigues me! I got the impression that the brain is somewhat well understood on the micro level (neuron-level, not neutron level) but that it all falls apart when trying to understand it on higher levels. (this book was ofcourse written decades ago)

Given current electroengineering technology and sub-infinite resources how slow/big would our model of the neural net of the brain become? The neurons in the model should reflect our current biological understanding of them.

size/speed is all free variables in this question. I reckon doing anything close to 86billion braincells got to be too slow/memory intensive for doing in software in general unspecialized supercomputers.

By speed i mean how long a 'thought' would take compared to a human if somehow the net was constructed and jumpstarted and it actually became an 'I'. How feasible would it be for a near future megacorp to afford to try making one?

Answer 1

Simulating human brain is currently absolutely not feasible for several reasons (I'll detail later).

AFAIK the only successful simulations (meaning: simulated result matched what biologic Neural Network actually did to a reasonable degree) were done for very simple organisms, if memory assists the most complex was a snail with several thousand neurons.

These simulations, however, were done simulating effects of neurotransmitters and neuron membrane; Simulated Neuron Networks (SNN, the kind of Neuron Networks that are actually in use to solve many problems, including prepare weather forecasts) work in a completely different way, the abstract the actual working of a neuron in a "stylized" way which has nothing in common with real Neuron operation. They are models of the neuron leaving out a lot of details to capture general principles of operation.

They do a good job, enough to be real useful in building A.I.

They're powerful enough to scare people like Elon Musk and Bill Gates with perspective results.

There's a heated debate if these SNN really capture enough of neurons working to permit replication of a complex brain.

In general problems arising when trying to simulate human brain are at several levels:

• Scale Problems
  • It is unclear if SNN really behave in a way comparable with biologic systems.
  • Very large SNN contain several hundred thousand Simulated Neurons our brain is close to hundred billion neurons.
  • Each SN has, at most, about 100 connections; typical neuron has more than 100 thousand connections.
• Physiological problems:
  • We do not exactly know how neurons are connected in our brain.
  • We have only a vague idea of function localization.
  • We have understood some of the interaction between neurons and neurotransmitters in blood stream.
  • We are recently starting to understand neurons not located in the brain (such as cardiac ganglia) have an important role in long-term memorization (thus giving a completely new meaning to the phrase "learn by hart").
  • Similar importance for overall brain operation have abdominal ganglia.
• Philosophical problems:
  • There still is no universal consensus (I have definite ideas on the subject though) if brain biochemistry can fully explain our subjective and objective behavior.
  • There still is no consensus of what actually is that we call "Conscience".
  • There still is no consensus if simulation can capture the relevant parts of what we cal "I".

The cited Hofstatter's book is a very interesting one and it is what spawned my interest on the subject, mut it is, IMHO, trying to hard to demonstrate a philosophical thesis ans thus it is, in the end, rather unconvincing.

Note: I summarized my personal understanding of the matter in a small site I wish I had more time to maintain. You'll find many references to published academic papers there.

Answer 2

Your modern computer is blindingly fast. A quad core 3Ghz i7 with hyperthreading does in excess of 12e9 computations per second. A GTX1080 GPU has ~2600 cores at ~1.6ghz, or 4,160e9 computations per second. Your brain, on the other hand, has 100e9 neurons all working in parallel. A neuron can fire about 200 times per second. giving us a rate of 20,000,e9 firings(?) per second.

So if we say that it takes 100 computer instructions (number pulled from hat) to simulate a neuron and ignore our ram requirements and lookup times. If we load that onto a single GTX 1080, and if we somehow assemble the neurons into a brain, how fast will it run?

On a single GTX1080, it will be 500 times slower ignoring ram lookups, assuming 100 instructions per neuron.

GTX1080 
      -> 4,000e9 instructions per second *  (1 artificial_neuron_triggers / 100 instructions)
      -> 40e9 artificial_neuron_triggers per second

HUMAN BRAIN
      -> 100e9 neurons * (200 neuron_triggers / 1 neuron) per second
      -> 20,000e9 neuron_triggers per second

---

So we're not that far off. Grab a couple hundred GPU's and you can be in the same order of magnitude. Why can't we (currently) simulate a brain in real time with a bunch of super-fast GPU's?

I can think of a few reasons (Also see ZioByte's answer):

1. We don't have a clue about how the brain fits together other than at the micro level (single neurons) and a touch at the macro level (eg MRI's), or if we do, I haven't heard about it (if you have, post links or edit this answer). As such, while we may be able to simulate networks of neurons, we (as far as I know) can't assemble them into a human brain.
2. A neuron is not a single calculation. A GPU is a vector processor, and just like your CPU, it can do things like adding numbers, or multiplying them. What does a neuron do? You can find pages and pages of math representing the behaviour of a single neuron. Needless to say, you will need an order of magnitude or more excess computation power to simulate an equivalent number of neurons. I assumed 100 instructions per neuron 'trigger', but I suspect that is far to low.
3. Memory lookups are really slow. If you have 100 billion neurons, you need 100 gigabytes of ram to store the system state (assuming a single byte per neuron - so you'll need more). While this is possible using caching to disk, access will be crazily slow. The slowest operation in most shaders in computer games is a texture lookup. I doubt this is any different for looking up neuron states. In a real brain each neuron stores it's own state, but we do not have 100 billion L1 caches (the really fast ones) on our GPU. We only have 2600 (one per core).
4. The architecture is all wrong. Even on a highly parelleized GPU, things are still synchronous, and your brain will be simulated in individual steps. So far as I know, a brain does not have a clock [citation needed] and is thus an asynchronous machine. It will be hard to change this in the near future, and I suspect that proper neural simulations cannot be done with current machine architectures.
5. Samwise points out in the comments that this ignores the role of synapses. These are likely to increase the required computation hugely.