<p>Check out:</p>
<p>1.6. Bayesian Hierarchical predictive coding approach to explanation of the findings</p>
<p>from this thesis:</p>
<p><a href="https://discovery.ucl.ac.uk/id/eprint/1435549/" rel="nofollow noreferrer">https://discovery.ucl.ac.uk/id/eprint/1435549/</a></p>
<p>There's a comparison between image compression algorithms and what the brain does.</p>

CC BY-SA 4.0
2015-12-15T11:09:13.863Z
2024-12-03T15:26:16.143Z
2024-12-03T15:26:16.143Z
Hauke Hillebrandt
<p>Say for example, you know what a particular type of flower looks like. You know it has purple petals and a white center. Now, unbeknownst to you, this flower has been slowly changing (evolving) to have a different shape, size, and even shade over the years since you first had seen one of these flowers. All the while, you have been around this flower your whole life. I mean, it grows in your current backyard, and it also grew in the backyard of the house you grew up in. In this scenario, you may have dreams about when you were a kid playing in the backyard. Instead of seeing what the flowers used to look like (when you actually <em>were</em> a kid), you will most likely see them as they appear to you today. Although your brain has memories of the <em>presence</em> of these flowers in the yard when you were a kid, their actual defining details have been rewritten over time with their current appearance. The hypothesis here is that your brain is trying to conserve capacity by linking all recorded encounters with this type of flower so that they all share the same "image file" in your brain. Witness testimonies have a similar problem -- where data received after the fact can change a person's memory of what happened and how things and people actually looked. I recommend you research the problems known about witness testimony.</p>

CC BY-SA 3.0
2015-12-15T16:07:29.930Z
Sounds like the brain does lossy compression
Chirag Bhatia - chirag64
CC BY-SA 3.0
2015-12-14T12:49:56.280Z
2015-12-14T12:49:56.280Z
Michael
<p>Horace Barlow was probably the first to elaborate on the idea of compression in sensory systems:</p>

<p>Barlow, Possible principles underlying the transformation of sensory messages. Sensory Communication, 1961.</p>

<p>It is interesting to get his perspective 40 years later from:</p>

<p>Barlow, Redundancy reduction revisited., Network, 2001.</p>

<p>More recently, Bialek and colleagues have done a lot of research on this (again, primarily in sensory systems). For example, here is a recent study where they quantify the amount of predictive information (about the future state of a stimulus) compressed into retinal ganglion signalling (and downstream):</p>

<p>Palmer et al., Predictive information in a sensory population, PNAS, 2015.</p>

<p>In this case, they observe that the retina encodes predictive information about the future state of an external stimulus at a rate that approaches a theoretic limit (according to their model based on the Information Bottleneck principle) thus exhibiting (roughly) maximal compression.</p>

CC BY-SA 3.0
2015-12-15T17:55:14.657Z
2015-12-15T17:55:14.657Z
daur
<p>The Semantic Pointer Architecture (SPA), which was used to built the biologically plausible cognitive-model Spaun, is a demonstration of the utility of compression in the brain.</p>

<p>In Spaun uses compression to relate concepts in a unified and grounded manner.  For example, for number concepts in Spaun's "copying task", Spaun first gets the concept of the number by gradually extracting larger and larger features until from a visual hierarchy. This is analogous to the Deep Learning's Restricted Boltzmann Machines. This compressed representation can than be decompressed into individual motor actions and sent back down the motor hierarchy.</p>

<p>To summarize, the perception problem is the dual of the action problem and they are unified my compression, as visualised in the image below taken from <a href="https://studywolf.wordpress.com/2014/07/" rel="nofollow noreferrer">the presentation found here</a>.</p>

<p><a href="https://i.sstatic.net/684QG.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/684QG.png" alt="perception problem is the dual of the action problem"></a></p>

<p>This compression has also been used in terms of <a href="https://cogsci.stackexchange.com/a/5738/4397">scalable knowledge representations</a>.</p>

<p>For further information on the concepts behind SPA and Spaun, beyond the context of compression, see "How to Build a Brain" by Chris Eliasmith.</p>

CC BY-SA 3.0
2016-03-07T00:27:57.590Z
2016-03-07T01:04:27.393Z
2017-04-13T12:56:16.467Z
Seanny123
<p>I am reading a book on information theory and one line in the book states that "Brains are the ultimate compression and communication systems." However, there is no further elaboration on the matter. I am curious to know what specifically this sentence may be referring to. In particular, how does the brain demonstrate compression?</p>

CC BY-SA 3.0
2016-03-07T01:05:36.540Z
@mrt I've added an answer relating the to Semantic Pointer Architecture that might interest you
Seanny123
CC BY-SA 3.0
2015-12-14T22:54:24.113Z
Sounds like semantic pointer architecture may be relevant (maybe)!
mrt
<theoretical-neuroscience>
In what ways do our brains use compression?
Sydney Maples
# In what ways do our brains use compression?

(Asked by: username_0 on 2015-12-14)

I am reading a book on information theory and one line in the book states that "Brains are the ultimate compression and communication systems." However, there is no further elaboration on the matter. I am curious to know what specifically this sentence may be referring to. In particular, how does the brain demonstrate compression?

**Tags:** theoretical-neuroscience

> **Comments:**
> **username_2**: Sounds like semantic pointer architecture may be relevant (maybe)!  
> **username_1**: @username_2 I've added an answer relating the to Semantic Pointer Architecture that might interest you  

---

## 4 answers

### Answer by username_3 on 2015-12-15. Score: 2

Check out:

1.6. Bayesian Hierarchical predictive coding approach to explanation of the findings

from this thesis:

<https://discovery.ucl.ac.uk/id/eprint/1435549/>

There's a comparison between image compression algorithms and what the brain does.


---

### Answer by username_4 on 2015-12-14. Score: 2

Say for example, you know what a particular type of flower looks like. You know it has purple petals and a white center. Now, unbeknownst to you, this flower has been slowly changing (evolving) to have a different shape, size, and even shade over the years since you first had seen one of these flowers. All the while, you have been around this flower your whole life. I mean, it grows in your current backyard, and it also grew in the backyard of the house you grew up in. In this scenario, you may have dreams about when you were a kid playing in the backyard. Instead of seeing what the flowers used to look like (when you actually *were* a kid), you will most likely see them as they appear to you today. Although your brain has memories of the *presence* of these flowers in the yard when you were a kid, their actual defining details have been rewritten over time with their current appearance. The hypothesis here is that your brain is trying to conserve capacity by linking all recorded encounters with this type of flower so that they all share the same "image file" in your brain. Witness testimonies have a similar problem -- where data received after the fact can change a person's memory of what happened and how things and people actually looked. I recommend you research the problems known about witness testimony.

> **Comments:**
> **username_5**: Sounds like the brain does lossy compression  

---

### Answer by username_6 on 2015-12-15. Score: 1

Horace Barlow was probably the first to elaborate on the idea of compression in sensory systems:

Barlow, Possible principles underlying the transformation of sensory messages. Sensory Communication, 1961.

It is interesting to get his perspective 40 years later from:

Barlow, Redundancy reduction revisited., Network, 2001.

More recently, Bialek and colleagues have done a lot of research on this (again, primarily in sensory systems). For example, here is a recent study where they quantify the amount of predictive information (about the future state of a stimulus) compressed into retinal ganglion signalling (and downstream):

Palmer et al., Predictive information in a sensory population, PNAS, 2015.

In this case, they observe that the retina encodes predictive information about the future state of an external stimulus at a rate that approaches a theoretic limit (according to their model based on the Information Bottleneck principle) thus exhibiting (roughly) maximal compression.


---

### Answer by username_1 on 2016-03-07. Score: 1

The Semantic Pointer Architecture (SPA), which was used to built the biologically plausible cognitive-model Spaun, is a demonstration of the utility of compression in the brain.

In Spaun uses compression to relate concepts in a unified and grounded manner. For example, for number concepts in Spaun's "copying task", Spaun first gets the concept of the number by gradually extracting larger and larger features until from a visual hierarchy. This is analogous to the Deep Learning's Restricted Boltzmann Machines. This compressed representation can than be decompressed into individual motor actions and sent back down the motor hierarchy.

To summarize, the perception problem is the dual of the action problem and they are unified my compression, as visualised in the image below taken from [the presentation found here](https://studywolf.wordpress.com/2014/07/).

[![perception problem is the dual of the action problem](https://i.sstatic.net/684QG.png)](https://i.sstatic.net/684QG.png)

This compression has also been used in terms of [scalable knowledge representations](https://cogsci.stackexchange.com/a/5738/4397).

For further information on the concepts behind SPA and Spaun, beyond the context of compression, see "How to Build a Brain" by Chris Eliasmith.


---

