I am currently reading an eternal golden braid, i've gotten as far as several lengthy chapters about the brain.

He talks about neurons and symbols, what 'meaning' is and there is a lot of hypothesizing going on. To me much seem like assumptions about our mind from a mind strongly affected by its knowledge about computers. Anyway it intrigues me! I got the impression that the brain is somewhat well understood on the micro level (neuron-level, not neutron level) but that it all falls apart when trying to understand it on higher levels. (this book was ofcourse written decades ago)

Given current electroengineering technology and sub-infinite resources how slow/big would our model of the neural net of the brain become? The neurons in the model should reflect our current biological understanding of them.

size/speed is all free variables in this question. I reckon doing anything close to 86billion braincells got to be too slow/memory intensive for doing in software in general unspecialized supercomputers.

By speed i mean how long a 'thought' would take compared to a human if somehow the net was constructed and jumpstarted and it actually became an 'I'. How feasible would it be for a near future megacorp to afford to try making one?

  • $\begingroup$ This looks like the closest we have (telegraph.co.uk/technology/10567942/…) $\endgroup$ – Pyrotrain Aug 25 '17 at 19:40
  • $\begingroup$ When you say "near future megacorp", do you mean a company somewhat like Google, IBM or Tesla? $\endgroup$ – a CVn Aug 25 '17 at 19:51
  • $\begingroup$ @Michael yes, pretty much like them $\endgroup$ – Adam Aug 25 '17 at 19:59
  • 1
    $\begingroup$ Neural nets have been studied mathematically for a long time. There are hard limits on what they can do. The brain is not structured like a neural net in the sense that this phrase has in computing. The brain does not work like a digital computer. A direct emulation of the brain is not technically feasible, because the brain is an analog structure made up of one hundred billion analog adders ("neurons") with many connections, and many more support "glial" cells of which the role is poorly understood. $\endgroup$ – AlexP Aug 25 '17 at 20:19

Simulating human brain is currently absolutely not feasible for several reasons (I'll detail later).

AFAIK the only successful simulations (meaning: simulated result matched what biologic Neural Network actually did to a reasonable degree) were done for very simple organisms, if memory assists the most complex was a snail with several thousand neurons.

These simulations, however, were done simulating effects of neurotransmitters and neuron membrane; Simulated Neuron Networks (SNN, the kind of Neuron Networks that are actually in use to solve many problems, including prepare weather forecasts) work in a completely different way, the abstract the actual working of a neuron in a "stylized" way which has nothing in common with real Neuron operation. They are models of the neuron leaving out a lot of details to capture general principles of operation.

They do a good job, enough to be real useful in building A.I.

They're powerful enough to scare people like Elon Musk and Bill Gates with perspective results.

There's a heated debate if these SNN really capture enough of neurons working to permit replication of a complex brain.

In general problems arising when trying to simulate human brain are at several levels:

  • Scale Problems
    • It is unclear if SNN really behave in a way comparable with biologic systems.
    • Very large SNN contain several hundred thousand Simulated Neurons our brain is close to hundred billion neurons.
    • Each SN has, at most, about 100 connections; typical neuron has more than 100 thousand connections.
  • Physiological problems:
    • We do not exactly know how neurons are connected in our brain.
    • We have only a vague idea of function localization.
    • We have understood some of the interaction between neurons and neurotransmitters in blood stream.
    • We are recently starting to understand neurons not located in the brain (such as cardiac ganglia) have an important role in long-term memorization (thus giving a completely new meaning to the phrase "learn by hart").
    • Similar importance for overall brain operation have abdominal ganglia.
  • Philosophical problems:
    • There still is no universal consensus (I have definite ideas on the subject though) if brain biochemistry can fully explain our subjective and objective behavior.
    • There still is no consensus of what actually is that we call "Conscience".
    • There still is no consensus if simulation can capture the relevant parts of what we cal "I".

The cited Hofstatter's book is a very interesting one and it is what spawned my interest on the subject, mut it is, IMHO, trying to hard to demonstrate a philosophical thesis ans thus it is, in the end, rather unconvincing.

Note: I summarized my personal understanding of the matter in a small site I wish I had more time to maintain. You'll find many references to published academic papers there.

|improve this answer|||||
  • $\begingroup$ Interesting! My angle towards the whole subject was more to give up understanding of how it might work at the high level and just model what we do understand as closely as possible. A.k.a. all the scale problems you mention would have to be solved. In the book Turing is mentioned talking about critical mass in regard to brain mass, maybe if an SNN with billions and billions of connectors were created, then maybe something unforseen would emerge $\endgroup$ – Adam Aug 27 '17 at 12:47
  • $\begingroup$ That is exactly what scares Musk, Gates and many others, including me. $\endgroup$ – ZioByte Aug 27 '17 at 12:51

Your modern computer is blindingly fast. A quad core 3Ghz i7 with hyperthreading does in excess of 12e9 computations per second. A GTX1080 GPU has ~2600 cores at ~1.6ghz, or 4,160e9 computations per second. Your brain, on the other hand, has 100e9 neurons all working in parallel. A neuron can fire about 200 times per second. giving us a rate of 20,000,e9 firings(?) per second.

So if we say that it takes 100 computer instructions (number pulled from hat) to simulate a neuron and ignore our ram requirements and lookup times. If we load that onto a single GTX 1080, and if we somehow assemble the neurons into a brain, how fast will it run?

On a single GTX1080, it will be 500 times slower ignoring ram lookups, assuming 100 instructions per neuron.

      -> 4,000e9 instructions per second *  (1 artificial_neuron_triggers / 100 instructions)
      -> 40e9 artificial_neuron_triggers per second

      -> 100e9 neurons * (200 neuron_triggers / 1 neuron) per second
      -> 20,000e9 neuron_triggers per second

So we're not that far off. Grab a couple hundred GPU's and you can be in the same order of magnitude. Why can't we (currently) simulate a brain in real time with a bunch of super-fast GPU's?

I can think of a few reasons (Also see ZioByte's answer):

  1. We don't have a clue about how the brain fits together other than at the micro level (single neurons) and a touch at the macro level (eg MRI's), or if we do, I haven't heard about it (if you have, post links or edit this answer). As such, while we may be able to simulate networks of neurons, we (as far as I know) can't assemble them into a human brain.
  2. A neuron is not a single calculation. A GPU is a vector processor, and just like your CPU, it can do things like adding numbers, or multiplying them. What does a neuron do? You can find pages and pages of math representing the behaviour of a single neuron. Needless to say, you will need an order of magnitude or more excess computation power to simulate an equivalent number of neurons. I assumed 100 instructions per neuron 'trigger', but I suspect that is far to low.
  3. Memory lookups are really slow. If you have 100 billion neurons, you need 100 gigabytes of ram to store the system state (assuming a single byte per neuron - so you'll need more). While this is possible using caching to disk, access will be crazily slow. The slowest operation in most shaders in computer games is a texture lookup. I doubt this is any different for looking up neuron states. In a real brain each neuron stores it's own state, but we do not have 100 billion L1 caches (the really fast ones) on our GPU. We only have 2600 (one per core).
  4. The architecture is all wrong. Even on a highly parelleized GPU, things are still synchronous, and your brain will be simulated in individual steps. So far as I know, a brain does not have a clock [citation needed] and is thus an asynchronous machine. It will be hard to change this in the near future, and I suspect that proper neural simulations cannot be done with current machine architectures.
  5. Samwise points out in the comments that this ignores the role of synapses. These are likely to increase the required computation hugely.
|improve this answer|||||
  • 2
    $\begingroup$ Some of the latest research is indicating that even every synapse can do useful computation, and seeing as there's ~10k synapses per neuron, that pushes up the limit by 10'000x (bringing up the limit to something like 10e18 calcs total), no wonder no-one's made a brain yet... $\endgroup$ – Samwise Aug 25 '17 at 23:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.