<p>Two ideas on this so far:</p>

<ul>
<li>I think we have neurons representing multiple occurrences of a given feature, for example one neuron for "one face", one for "two faces", etc. At some number it doesn't really make a big difference anymore, so there is just a neuron for "group of faces". This would explain why we can recognize small number of objects in a glimpse but have to count for larger amount. Corrections and suggestions are welcome.</li>
<li>In his 2014 talk at MIT, Prof. Hinton explains how the brain might find equivariant features instead of invariant features. Basically, in addition to existence of a feature, neurons in the same cortical cell might output the properties of this feature. In vision, those properties would be position and rotation, for example.</li>
</ul>

CC BY-SA 3.0
2016-01-29T18:08:30.640Z
2016-02-06T00:04:55.740Z
2016-02-06T00:04:55.740Z
danijar
<p>You are basically asking how to bind different concepts together based off their representation in neurons. The one way I know how to do this is using the Semantic Pointer Architecture (SPA).</p>

<p>To understand how lower level neural activities can be compressed as a concept, please see my answer on <a href="https://cogsci.stackexchange.com/a/13487/4397">compression in the brain using the SPA</a>. This answer explains how low-level features can be compressed into a higher level concept vector.</p>

<p>Once you've accepted, that concepts can be represented as high dimensional vectors and these vectors can be manipulated using neurons, we now have to define what types of manipulations can be used to represent counts.</p>

<p>A perfect example of this is <a href="http://compneuro.uwaterloo.ca/files/publications/rasmussen.2010.pdf" rel="nofollow noreferrer">Dan Rasmussen's neural system for solving Raven's Progressive Matrices</a>. In the system, graphical cells in a matrice are represent using a neural symbols bound together using circular convolution. For example, the top-left cell in the below image could be represented as symbols bound together as <code>SHAPE*CIRCLE+COLOR*BLACK+NUMBER*ONE</code> where every word written in ALL CAPS is a symbol represented by a high dimensional vector and based on sensory input.</p>

<p><a href="https://i.sstatic.net/VUrSF.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/VUrSF.png" alt="matrix of symbols"></a></p>

<p>To relate numbers to each other, the number <code>ONE</code> is defined as a unitary vector which can be convolved with itself to represent new numbers. For example <code>TWO=ONE*ONE</code> and <code>THREE=ONE*ONE*ONE=TWO*ONE</code>.</p>

<p>For implementation details and to try out these concepts for yourself if a simulator, please see <a href="https://github.com/nengo/nengo" rel="nofollow noreferrer">Nengo</a> and "How to Build a Brain" by Chris Eliasmith.</p>

CC BY-SA 3.0
2016-03-07T21:34:43.080Z
Interesting mechanic. Would that also account for multiple types of objects and their positions and counts (assuming the brain can do that which is not clear)?
danijar
CC BY-SA 3.0
2016-03-08T00:37:00.383Z
The example I gave only covered the case of Raven Progressive Matrice, but hypothetically any structure is possible given a high enough dimensional vector. This raises two sub-questions: 1. What is the limit of the dimensions that the brain can represent? 2. How can we define this in a general manner in the vision system. For the first question, this is discusses somewhere in "How to Build a Brain", but it was a pretty large number, like 4096 if I'm not mistaken. For the second question, it depends on the vision architecture.
Seanny123
CC BY-SA 3.0
2016-03-08T00:39:42.930Z
It's easy to imagine changing the receptive fields of a spiking version of a convolutional neural net depending on the task so that various structures can be extracted. However, implementing it is non-trivial.
Seanny123
CC BY-SA 3.0
2016-03-07T01:21:47.360Z
2016-03-07T01:21:47.360Z
2017-04-13T12:56:16.167Z
Seanny123
<p>I understand that levels in the sensorimotor hierarchy of the brain learn recurring features of their input streams. The results are activation patterns with single neurons firing for the existence of specific higher level patterns. Those sparse representations are passed upward to the next higher layer.</p>

<p>For example, there is a neuron firing whenever there is a face in the scene. On this higher level, neurons are not restricted to a small window of our field of view anymore. My question is how those high level sparse representations still retain information about positions and counts of objects.</p>

<p>One bad explanation would be that there are neurons representing counts in general. If "three" and "face" are both active, there are three faces in the scene. But this doesn't hold for two kinds of objects at the same time since it's not clear which count is for which object. And what if both have the same count.</p>

CC BY-SA 3.0
2016-01-28T19:12:17.973Z
Your title seems to be missing some words: "How are positions and counts of higher _____ _____ encoded in sparse representations?" I've also read the description of your question, and, unfortunately, I cannot answer it. However, I'd suggest that representations are always distributed, so the position information may be at a lower level than the identity representation. Counting visual objects, on the other hand, requires an eye-motion algorithm (except in the case of the acquired skill of subitizing) based partly on the skill of pure, abstract counting. Where's the result stored?
John Pick
CC BY-SA 3.0
2016-01-28T19:18:12.400Z
Thanks for pointing out. Also, it's a good insight that counting is a temporal process. And for very small numbers (that can be recognized at a glimpse) that might indeed be individual neurons representing the numbers. So the problem is more about positions. I have the impression that positions are very temporal, too. Do you know if the "face" neuron would fire only if there is a face right in front of you, i.e. in the center of the field of view where we have sharp vision?
danijar
CC BY-SA 3.0
2016-01-29T05:15:42.820Z
I'd expect the "face" neuron to fire if you were thinking of the appearance of a face, whether you could actually see it or not. But, I think you are wondering if the "face" neuron can fire when a face is detected only in periphery vision (and there were no other clues such as a voice or an attached body that would evoke "face"). I guess it would depend on degree of eccentricity of gaze, whether that part of the periphery has high enough resolution, and whether you have enough experience recognizing faces in that part of your periphery.
John Pick
CC BY-SA 3.0
2016-01-29T18:05:44.063Z
My question was not if periphery vision is enough to recognize faces. I though that maybe by design, the "face" neuron only fires when there is a face in the center of view span. This would explain how two faces would be represented, that is, they won't be represented at the same time at all.
danijar
<cognitive-neuroscience><cognitive-modeling><sensory-perception>
How are positions and counts of higher concepts encoded in sparse representations?
danijar
# How are positions and counts of higher concepts encoded in sparse representations?

(Asked by: username_0 on 2016-01-28)

I understand that levels in the sensorimotor hierarchy of the brain learn recurring features of their input streams. The results are activation patterns with single neurons firing for the existence of specific higher level patterns. Those sparse representations are passed upward to the next higher layer.

For example, there is a neuron firing whenever there is a face in the scene. On this higher level, neurons are not restricted to a small window of our field of view anymore. My question is how those high level sparse representations still retain information about positions and counts of objects.

One bad explanation would be that there are neurons representing counts in general. If "three" and "face" are both active, there are three faces in the scene. But this doesn't hold for two kinds of objects at the same time since it's not clear which count is for which object. And what if both have the same count.

**Tags:** cognitive-neuroscience, cognitive-modeling, sensory-perception

> **Comments:**
> **username_1**: Your title seems to be missing some words: "How are positions and counts of higher _____ _____ encoded in sparse representations?" I've also read the description of your question, and, unfortunately, I cannot answer it. However, I'd suggest that representations are always distributed, so the position information may be at a lower level than the identity representation. Counting visual objects, on the other hand, requires an eye-motion algorithm (except in the case of the acquired skill of subitizing) based partly on the skill of pure, abstract counting. Where's the result stored?  
> **username_0**: Thanks for pointing out. Also, it's a good insight that counting is a temporal process. And for very small numbers (that can be recognized at a glimpse) that might indeed be individual neurons representing the numbers. So the problem is more about positions. I have the impression that positions are very temporal, too. Do you know if the "face" neuron would fire only if there is a face right in front of you, i.e. in the center of the field of view where we have sharp vision?  
> **username_1**: I'd expect the "face" neuron to fire if you were thinking of the appearance of a face, whether you could actually see it or not. But, I think you are wondering if the "face" neuron can fire when a face is detected only in periphery vision (and there were no other clues such as a voice or an attached body that would evoke "face"). I guess it would depend on degree of eccentricity of gaze, whether that part of the periphery has high enough resolution, and whether you have enough experience recognizing faces in that part of your periphery.  
> **username_0**: My question was not if periphery vision is enough to recognize faces. I though that maybe by design, the "face" neuron only fires when there is a face in the center of view span. This would explain how two faces would be represented, that is, they won't be represented at the same time at all.  

---

## 2 answers

### Answer by username_0 on 2016-01-29. Score: 1 | Accepted answer

Two ideas on this so far:

* I think we have neurons representing multiple occurrences of a given feature, for example one neuron for "one face", one for "two faces", etc. At some number it doesn't really make a big difference anymore, so there is just a neuron for "group of faces". This would explain why we can recognize small number of objects in a glimpse but have to count for larger amount. Corrections and suggestions are welcome.
* In his 2014 talk at MIT, Prof. Hinton explains how the brain might find equivariant features instead of invariant features. Basically, in addition to existence of a feature, neurons in the same cortical cell might output the properties of this feature. In vision, those properties would be position and rotation, for example.


---

### Answer by username_2 on 2016-03-07. Score: 1

You are basically asking how to bind different concepts together based off their representation in neurons. The one way I know how to do this is using the Semantic Pointer Architecture (SPA).

To understand how lower level neural activities can be compressed as a concept, please see my answer on [compression in the brain using the SPA](https://cogsci.stackexchange.com/a/13487/4397). This answer explains how low-level features can be compressed into a higher level concept vector.

Once you've accepted, that concepts can be represented as high dimensional vectors and these vectors can be manipulated using neurons, we now have to define what types of manipulations can be used to represent counts.

A perfect example of this is [Dan Rasmussen's neural system for solving Raven's Progressive Matrices](http://compneuro.uwaterloo.ca/files/publications/rasmussen.2010.pdf). In the system, graphical cells in a matrice are represent using a neural symbols bound together using circular convolution. For example, the top-left cell in the below image could be represented as symbols bound together as `SHAPE*CIRCLE+COLOR*BLACK+NUMBER*ONE` where every word written in ALL CAPS is a symbol represented by a high dimensional vector and based on sensory input.

[![matrix of symbols](https://i.sstatic.net/VUrSF.png)](https://i.sstatic.net/VUrSF.png)

To relate numbers to each other, the number `ONE` is defined as a unitary vector which can be convolved with itself to represent new numbers. For example `TWO=ONE*ONE` and `THREE=ONE*ONE*ONE=TWO*ONE`.

For implementation details and to try out these concepts for yourself if a simulator, please see [Nengo](https://github.com/nengo/nengo) and "How to Build a Brain" by Chris Eliasmith.

> **Comments:**
> **username_0**: Interesting mechanic. Would that also account for multiple types of objects and their positions and counts (assuming the brain can do that which is not clear)?  
> **username_2**: The example I gave only covered the case of Raven Progressive Matrice, but hypothetically any structure is possible given a high enough dimensional vector. This raises two sub-questions: 1. What is the limit of the dimensions that the brain can represent? 2. How can we define this in a general manner in the vision system. For the first question, this is discusses somewhere in "How to Build a Brain", but it was a pretty large number, like 4096 if I'm not mistaken. For the second question, it depends on the vision architecture.  
> **username_2**: It's easy to imagine changing the receptive fields of a spiking version of a convolutional neural net depending on the task so that various structures can be extracted. However, implementing it is non-trivial.  

---

