How much memory is needed to record a human thought?

Question

Mind-Machine interfacing is fairly common across the width and breadth of sci-fi, from Science-fantasy to Hard Science-Fiction. The idea of connecting a human mind to a computer system especially the ability to record human thought and experience got me thinking.

How much information is contained inside a single human thought or memory?

And.

How many bytes are needed to record even the simplest of ideas and concepts?

Related to the question above is another "do thoughts/ideas have a quantifiable size and if so what is their unit of measure called?".

Answer 1

Well a concept is a network of associations, the foundation of this network are your sensory inputs (or whatever interpretation your brain receives, a lot of pre-processing occurs in the nervous system) for example icecream is defined by its temperature, appearance, smell, flavour, all the properties that come to mind when you think of the word "icecream".

The word itself is quite an abstract concept, an even more abstract concept would be strawberry icecream which is built upon both the foundations of icecream and strawberries. Of course the concept of sweetness isn't unique to either the icecream or strawberries so these concepts are already partly entangled before the association that forms the strawberry-icecream concept takes place.

So measuring the memory capacity of an associative conceptual network is incredibly difficult because many conceptual structures share common foundations and we don't even know how deep these foundations go, e.g. what is the bare minimum of information required to accurately recognise ice-cream?

Judging by what I've read about people with alzheimer's disease I think it's possible for a person to lose a great deal of their memory and still remain a functional sentient consciousness, though admittedly a crippled one, and it's anyone's guess what the bare minimum amount of memory is.

Answer 2

Hopfield Networks

Your best bet is to look at Hopfield networks. They are auto-associative artificial neural networks (ANNs) which model some aspects of human memory. That is, the behavior of Hopfield networks under certain learning rules is similar to the performance of certain human memory (the kind that would end up in the frontal cortex as opposed to, say, the cerebellum).

Like human memory, Hopfield networks do not exhibit "perfect" recall for all input vectors. However, the recall will be strongest when the input is closest to the recall vector. Sometimes, a Hopfield network will converge to the "opposite" of the intended recall vector (something which doesn't seem to happen to humans, possibly because of additional filtering circuitry rejecting such recalls...or perhaps, this does happen, but manifests itself in subtle ways).

The size of the vector stored in a Hopfield net is equal to the number of nodes in the network. That is, the nodes themselves represent both the inputs to the memory and the manifested outputs. This may or may not be similar to the way parts of the human brain are organized. If it is indeed similar, then we can derive a few bits of performance data.

Performance

The number of "storable" vectors in a Hopfield network appears to be about 14%. So let's say you have a network which stores the names of people and places you know. In order to reliably store 1000 names, you would need about 7000 neurons. Pretty good, huh? Now imagine you need to store their faces. Uh-oh. The human retina has more than 3 million photoreceptors, which is about 1 million pixels, accounting for color. If you tried to store images at full resolution, then, you would need a network of 3 million neurons. Then you could store about 400k distinct images (assuming the vectors are sufficiently different). However, it is very likely that visual memories are not stored in the brain in this way, due to the fact that the higher brain only has access to the output of the visual cortex, which does a tremendous amount of pre-processing (edge and segment detection along multiple angles, movement detection, etc.).

So how many neurons do we have to work with? The raw total for humans is about 86b. But much of the brain is control circuitry (for operating your automatic functions like heart rate, digestion, etc., skeletal muscles, other organs). If we limit ourselves to the cerebral cortex, which is where "higher order memories" are most likely to be stored, we are looking at about 21b neurons, or about a quarter of the total.

Definitions

At this point, it could be tempting to work backwards, and say that the human brain could then store about 3 billion memories. But remember that a Hopfield network stores vectors which are the length of the number of nodes in the network, so these "memories" would be ~3 GB each!!! And we know by the wiring of the cerebral cortex that it cannot possibly be a single Hopfield network. A Hopfield network is also fully connected (every node connects to every other), and the cortex is highly layered. On average, human neurons connect to about 7000 neighbors. Thus, if the brain contains any Hopfield networks at all, they are likely to be fairly small.

In the limiting case where the entire cortex is composed of Hopfield nets (not plausible), we would have ~3 million networks which can each store about 1000 vectors of ~1 KB each. Although that is still 3 billion total vectors, we now have the challenge of mapping these vectors onto memories. We must thus ask: "What is a memory?" The fact that Americans celebrate Thanksgiving day in November might be considered a memory. And the fact that they tend to travel to be with family might also be considered a memory. But what about last Thanksgiving? Is the smell of turkey a memory? How about the smell of turkey + sweet potatoes + apple pie? Are those distinct memories or pieces of the same memory? Is the football game part of the "last Thanksgiving" memory, or its own memory?

The fact that memories are inherently fuzzy does't help matters any. They do not have crisp boundaries, and they can be hierarchical. Can you remember which part of the bird you ate? Light meat or dark? Which family members were present? What they said? How many of these questions are answered because you stored discrete facts vs. recalled an image and queried it?

For these reasons, the very question of memory storage density is ill-posed. But if we agree that there is some smallest unit of recall for the human brain, then this would almost surely correspond to a vector in a Hopfield-like network. And as you can see above, the upper bound for those units is about 3 billion, for the average human brain. It may be that images require many such vectors to store, and that they are always stored with many vectors, making the total number of distinct "memories" much smaller. And some folks may object that humans can "only" store 3 billion distinct items in their head. So let me address that briefly.

Implicit Knowledge

Do you play tennis? How about ping pong? Can you make a complete novice into a good player using just words? No. At the very least, the novice must actually "go through the motions". The programming doesn't happen in the ears. It happens in the cerebellum. And while some folks will think of bodily-kinesthetic programming as a functional wiring of motor control circuits, there is definitely a memory capability involved. Someone who has learned to play tennis will have a leg up learning ping pong, and vice versa. That is partly because the control circuits will be wired to make similar motions, but also because the players will have memories of specific trajectories and responses which are activated in particular circumstances.

When new players play against each other for the first time, they will often perform worse than they would against a familiar person of the same skill level. That is because skill is ultimately an ad-hoc covering of the state space for the game. If the other player spins the ball or attacks in a way that hasn't been seen before, your control circuits will not have a pre-made response, even if you were physically capable of one. The conscious brain is too slow to act decisively in competitive sports. Even if you know the appropriate response cognitively, if the cerebellum has not executed the program which covers that part of the state space (including your relative position, balance, momentum, etc.), you will likely fail to produce an adequate response. Much of competitive sports boils down to remembering the best move for a particular state. Low-level programs control fine details like which muscle fibers to activate, but high-level programs like "move right while swinging across the body" must be practiced to store to memory so that it can be activated automatically in the right circumstances.

These kinds of memories generally cannot be named, and would not be thought of as discrete. They are implicit in the programming which constitutes "athletic skill". In the same way, verbal behavior can also be implicit. For instance, most English speakers can finish the phrase: "See you ____." They will generally not say "catfish" or "pulverize" or "flavorful". The fact that most speakers will finish the phrase in the same or similar way means that this bit of behavior has more to do with the mechanics of language than the personal memories of the speaker. Thus, this information is likely not stored in the pre-frontal cortex (because we know that language facility is primarily handled by Broca's and Wernick's areas). In the same way, athletic "memory" is most likely stored in the cerebellum.

I presume that you mean to exclude these kinds of "implicit memory" in your calculations. If so, then limiting the analysis to the PFC is appropriate. Otherwise, you will also need to consider the "functional" areas in the remaining portions of the brain, which is much more difficult, given that we don't have any really good models of how these work.

Conclusion

So, I would say that it's safe to assume that humans can "remember" much less than 3 billion distinct [personal] memories, and that the smallest chunks are on the order of 1 KB of information. That puts an upper bound of about 3 TB of information that makes you a unique history of a human. Sobering thought, huh?

Answer 3

You can not possibly measure how much storage one thought needs.

In general, you need to separate three layers:

1. reality
2. abstraction
3. representation

Take something you have on your desk, for example a pen. Thats a real object. Something you can hold in your hand. Now your brain will identify the object as the concrete pen you own, but will also recognize the abstract concept of what a pen is (pen, stift, stylo, bolígrafo). In this answer, I use the English language to convey that thought, and use three glyphs "p", "e" and "n" to form its written representation.

Now think about something trivial. "My pen is red". You will see that it is nearly impossible to determine how much information you need to store for that thought. You need to store the idea of what a pen is. You need to store the idea of what a color is. Can someone who is blind ever understand what a color is when loading that thought into his brain? In order to convey the idea of what a pen is, what information is needed? If you write down everything you know about a pen, the list gets exhausting very fast.

The human brain is capable of sorting things and categorizing things based on your whole cumulative knowledge. Without all your background knowledge also being stored, the single thought wouldn't have any meaning. But exactly quantifying what part of your background knowledge is needed to store that single thought is almost impossible, because all our knowldege is incredibly inter-linked.

Answer 4

See my answer here for background and other links.

Read How to Create a Mind: The Secret of Human Thought Revealed by Ray Kurzweil, for general ideas on what you're getting into.

Now that you know about the individual pattern-matching units making up the neocortex, a thought is a pattern of activation of triggering the matchers up and down the chain. Now, they don’t have UUIDs or any labels, and they don't mean anything other than the context of what they are connected to.

So if you numbered them, and noted the activation particulars, that would represent the thought. It's only meaningful in the context of the brain it was read from, and that changes over time: you can’t play it back later because the specific pattern matchers have changed their topology and detailed weightings, so the “same” pattern matching unit has a slightly different meaning if stimulated again in the same way.

Recording thoughts means using language, which is something the brain is already wired for. Only consider a language that is far more precise than any normal human language, and a custom system to decode it into the context of the current brain receiving it. This system would need to be more powerful and hold more data than a human brain! And the result would still be imperfect.

One of my favorite books, GEB has some lengthy treatment on mapping one network to another. If you have an epub, search for “USA ASU”: that’s chapter 12 I see.

---

For use in a story, I'm considering how to efficiently copy deltas back to your home copy or, more ambititously, how to merge copies. I think this can be done with logfiles. The particular pattern-matching instance only means something relative to the brain it is found in. But if you did give it an ID, and then logged the changes to the input and output weights and any new hookups, that could be played back to an earlier snapshot to bring it up to date, and merging (with a little intelligence) would make sense if they have not made large changes.

So, how many pattern primitives change and at what rate, as you gain experience and think about things? I don’t know, but that’s a place to start looking for hard numbers.

Answer 5

Depends on the person, on your compression algorithm, and the particular thoughts you're recording.

But, to avoid answering with just "I don't know" - written English apparently contains about 1-1.2 bits of information per letter, which suggests that that much of thought, at least, won't take up too much space (then again, if we guess at four letters per word and one word per second, eight hours a day of thought and a hundred eighty days of schooling a year, K-12 education takes up ~65 MB). I'm using per-letter rather than per-thought size here because it seems obvious that thoughts can vary in size by an order of magnitude (contrast "the sky is red" with Dickens' opening sentence in "A Tale of Two Cities").

The unconscious framework those words are constructed on must also take up some space - written English intuitively should provide close to zero bits of information per letter to someone who only speaks, say, Cantonese. I'd expect that could be calculated based on the amount of time taken to learn the language in question (a few years for most children, and I think similar for adults if you ignore accents), times the amount of bandwidth devoted to learning it (which is itself a couple of unknown-to-me's in its own right: maximum human aural/visual bandwidth, and percentage of it spent listening/reading).

Answer 6

Maybe you should instead be asking: how much memory is needed to record a human thought that is minimally intact.

As an example, how would one record the thought "the sun is hot."?

Upon thinking of that single string, I come to a plethora of thoughts, including "I need to wear sunscreen" and "Carl Sagan said 'we are starstuff, pondering the stars'" and "stars go supernova which are very very hot but not as hot as regular stars which are still very hot."

It all depends on the context in which you need to access the thought. If it is something that can be broken down into discrete units like mathematical equations such as y = ax + b, then there is little relational data needed to compartmentalize the thought, whereas a larger thought, like "if I throw this baseball forward how far will it go until it hits the ground?" requires several data sources and interpretations, from the sheer physics (angle and velocity and gravity) to the personal (am I rested? am I sore from exercising yesterday? Do I care enough to throw this baseball as far as I can or do I want to pick it up without walking too far?). Even then, there may be tons of irrelevant information floating around regardless, like a memory of a movie about baseball with a scene that has little to do with baseball itself.

So in the end, the issue is less about storage space and more about bandwidth.

Answer 7

To be a bit pedantic, the simplest thought I can come up with is the number 1. It takes a single bit to store that thought, so there's your answer - one bit.

Anything beyond that, though, and you bump into how fundamentally unanswerable this question is. It's like asking how long a book is. Well, it entirely depends on the book. The world's longest book contains almost 2 million words long, and the shortest books in the world literally contain no text (if you don't like that, you can find any number of children's books that contain dozens of words).

So it is with thoughts. It's possible to conceive of a thought that takes only a single bit to store, and it's possible to conceive of a rambling stream of consciousness that would require vast arrays of storage media to completely record.

Answer 8

Human brains are estimated to be between 10TB and 100TB.

The calculation to get that number is actually simple:

100 billion neurons total * 1,000 connections (potential synapses)

As others have said, in reality it is not that simple. There are a lot of other factors that would affect that count:

• Do some synapses transmit with varied strength? (more data)
• Are multiple synapses sometimes needed to convey a single piece of information? (less data)
• Are some synapses used for processing, not storage? (less data)
• Are support cells also used to store data (more data)

Realistically, memories, especially complex memories would be hard, if not impossible to extract from the rest of the brain. The most accurate and lossless way to store memories would involve storing the entire brain's knowledge, and a map of what synapses were active.

Actually, as long as the brain is available, scientists already have the ability to use brain stimulation to essentially store a pointer to a memory:

An unexplained phenomenon in neuroscience is the discovery that electrical stimulation in temporal neocortex can cause neurosurgical patients to spontaneously experience memory retrieval

I'm not sure how much memory would be needed to store it, but it's probably not a huge amount. It's the most realistic example you will find.

Answer 9

Since science doesnt understand the human brain to a high degree, its hard to say.

Long-term memories are stored throughout the brain as groups of neurons that are primed to fire together in the same pattern that created the original experience, and each component of a memory is stored in the brain area that initiated it.

There are approximately 100 billion (100,000,000,000) neurons in the human brain.

If each neuron was equated to a computer bit, that would equal 12.5 gigabytes. So that means all your memories would fit on a BluRay DVD, along with 3 other people's.

But memories arent permanent, they can be replaced. Not only that, memories are known to leak into each other. The human memory is very fluid.

Answer 10

Before you can answer this question, you would have to define what you mean by the word "thought". Only then can you proceed, because otherwise you could easily be out by orders of magnitude. (One order of magnitude is a factor of 10).

According to Hutter's AIXI theory, intelligence is essentially equivalent to data compression. Let's assume that the human brain optimally compresses thoughts for most efficient storage. There is some evidence for this since research has shown that we only save the "gist" of a story rather than the details, for example. On that basis we could postulate that the smallest possible thought might be as small as one bit or just a few bits of information.

It's also possible to imagine situations where a thought might comprise of mix of images, sounds, smells, feelings, beliefs, etc. all interlinked. Would these be regarded as many linked thoughts or one super-thought? If the latter, it's easy for me to imagine that the storage requirement for this would in the order of hundreds of megabytes at least.

Answer 11

This question is incredibly interesting, but this has a very complicated answer beyond what some people would consider.

Consider this for a memory, you are walking down a road in a park and see some beautiful trees and your friend Greg says hello. What all is included in these details?

1. Your friend and all details associated with him
2. The trees and what they have looked like
3. The road you're walking down
4. The park and all details about it you remember

The next level of detail beyond what I've mentioned comes in when we include the neurochemical responses from the brain that we don't fully understand yet. For example, does our brain remember that the Alpha 2A receptor was stimulated in response to a medication or an "exciting" stimuli.

This comes to the question, does our brain store the chemicals that are produced in a memory, and if so is that how we remember how we've felt when remembering something. Additionally, if it does store these chemicals are they reproduced when remembering something?

If they are in fact reproduced when remembering something, is it possible that our brain remembers the chemicals and bits of the memory are stored in long term fully and the rest is fragmented and using associative memory our brain pieces the rest together.

In short I don't believe that we can even come close adequately stating how many bytes are stored in a memory, because our fundamental understanding of memory/brain neurochemical pathways are limited.

These are my thoughts on the question!

Answer 12

the size of a thought will be dependent on the number of combinations of thougts of equivalent size

Take an integer.

Most integers in computers are 32 bits. There are 2^32 combinations.

Now take a thought. Are there ten thoughs of the same size? Twenty? Maybe it is 1000000? Unfortunately, thoughts are too abstract. Therefore, we cannot determine their minimal size for storage.

but what about actual word thoughts or sensory data?

If we limit our scope to that, then the size is the size of a string (array of characters), and the size of audio/visual/scent/taste/touch data. The first two will be roughly a gigabyte per hour at hd quality. The other three are far too bizarre. I will leave those for when smellovision, tasteovision, and touchovision come out.

I cannot wait to see a zombie movie on smellovision!

Answer 13

A question to consider before this question can be answered: do people in your world have a strong understanding of the processes required for human cognition to the level where they can be simulated at a rate substantially faster than realtime? If they don't, the only way a thought can be captured is by exactly mapping the neural connections and activation state of every part of the thinker's brain (and probably also their connected nerve systems, so that the thought isn't interrupted by a discontinuity of sensations when you try to replay it). This is, as other answers have stated, a vast quantity of data (10^14 connections between 10^10 neurons with substantial quantities of data required about each connection and neuron's state to have a chance of meaningful replay -- say at an absolute minimum 10^16 bits of information or somewhere in the vicinity of a petabyte, but probably much more).

If, on the other hand, you have available the ability to rapidly simulate a brain state, it would be possible to simplify the current brain state by pruning parts of it and see whether or not they qualititively effect it. This may be able to take your petabyte of information and squeeze it down to something somewhat more manageable -- this would be a lossy compression process, but you should be able to work it so that what is lost is not relevant to the thought you were trying to encapsulate.

That said, if you have this capability, you've already fallen into a technological singularity, so pretty much all bets are off in terms of plausibility or otherwise of just about any technology.

Answer 14

I think the closest realistic estimate may be to compare Deep Blue to human capacity. It's not a single thought, but it is a single functionality (chess playing), and was built to simulate human calculation. Of course there are more complex thoughts and memories, but this may come closer to what the OP was asking if you divide the hardware firepower down from its max capacity with the hardware it was built with. https://en.wikipedia.org/wiki/Deep_Blue_(chess_computer) Alternatively, you can look at chess software and see how large those programs are and extrapolate from there.