<p>The human visual processing system receives input from the eyes, and then passes it through a number of areas of the brain that break it down, process it in various different ways, recombine it, and break it down again several times.  I'm assuming this question is only about the visual cortex, general theories about how information might be broken down for processing there, and only concerning parallel-processing (channels), as opposed to sequential specialization (layers), otherwise the answer should fill up a textbook.</p>

<p><strong>Main pathways through visual cortex:</strong></p>

<p>The <a href="http://en.wikipedia.org/wiki/Visual_cortex#Introduction" rel="nofollow noreferrer">traditional understanding</a> of the visual cortex divides it into <a href="http://en.wikipedia.org/wiki/Two-streams_hypothesis" rel="nofollow noreferrer">2 main pathways</a>:</p>

<blockquote>
  <p>Each V1 transmits information to two primary pathways, called the
  ventral stream and the dorsal stream.</p>
  
  <ul>
  <li><p>The ventral stream begins with V1, goes through visual area V2, then through visual area V4, and to the inferior temporal cortex (IP
  cortex). The ventral stream, sometimes called the "What Pathway", is
  associated with form recognition and object representation. It is also
  associated with storage of long-term memory.</p></li>
  <li><p>The dorsal stream begins with V1, goes through Visual area V2, then to the dorsomedial area (DM/ V6) and Visual area MT (middle
  temporal/ V5) and to the posterior parietal cortex. The dorsal stream,
  sometimes called the "Where Pathway" or "How Pathway", is associated
  with motion, representation of object locations, and control of the
  eyes and arms, especially when visual information is used to guide
  saccades or reaching.</p></li>
  </ul>
</blockquote>

<p>The "what" and "where" pathways are also sometimes referred to as the "perception" and "action" pathways, respectively.</p>

<p><img src="https://i.sstatic.net/bBCa2.jpg" alt="enter image description here"></p>

<p>Another <a href="http://en.wikipedia.org/wiki/Visual_modularity" rel="nofollow noreferrer">common theory</a> about the visual cortex identifies at least 3 modules: Motion, colour, and form.  Evidence for these proposed modules comes largely from lesion studies and related disorders of the visual system.  In general, an impairment in one module doesn't seem to affect the others, suggesting that these specializations are processed independently and in parallel.  However, it is not clear if they form distinct pathways through the visual cortex.</p>

<p>There is also evidence for a <a href="http://onlinelibrary.wiley.com/doi/10.1002/hipo.20320/abstract;jsessionid=60AB8D41D856EA7A87DBEBEAEB81F9C9.f03t03" rel="nofollow noreferrer">long-term memory pathway</a> (visual-perirhinal-hippocampal stream) that feeds memory information from and to the visual cortex.  Many other pathways have been postulated; this field of research is very active, consensus on further detail remains elusive for now.</p>

<p><strong>Complications to such models:</strong></p>

<p>Such "pathway" models of the visual cortex are limited by a number of factors:</p>

<ul>
<li>There is evidence of <a href="http://www.sciencedirect.com/science/article/pii/S0959438898800421" rel="nofollow noreferrer">significant interaction</a> between the streams</li>
<li>There is evidence of <a href="http://www.ncbi.nlm.nih.gov/pubmed/20007766" rel="nofollow noreferrer">feedback cycles</a> within the streams</li>
<li>Evidence that <a href="http://www.jneurosci.org/content/21/24/9844.short" rel="nofollow noreferrer">different pathways</a> are used in different situations</li>
<li>There is some evidence of <a href="http://www.ncbi.nlm.nih.gov/pubmed/8787209" rel="nofollow noreferrer">asymmetric processing</a> in the brain</li>
<li>There is also some evidence of <a href="http://www.ncbi.nlm.nih.gov/pubmed/17338604" rel="nofollow noreferrer">inter-person differences</a></li>
</ul>

<p>Furthermore, as mentioned, the visual cortex is only part of the visual system.  Significant processing happens earlier - you mentioned about edge detection, some of that appears to already be happening by the time signals reach the visual cortex.  And more processing happens later as well, in the parietal and temporal cortices for example, such as facial recognition.</p>

<p><strong>Implications for artificial neural networks:</strong></p>

<p>The visual cortex may not be ideal inspiration for ANNs, especially since it only handles a not-very-early part of the process, such that the signals it receives have already undergone significant modification from the raw stimuli.  Additionally, its pathway structure is fairly complex, and not fully understood.  There have nonetheless been <a href="http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4018419" rel="nofollow noreferrer">attempts to simulate</a> parts of the visual cortex and compare to non-biologically inspired networks.</p>

<p>The <a href="http://en.wikipedia.org/wiki/Lateral_geniculate_nucleus" rel="nofollow noreferrer">LGN</a> - an area of the brain involved earlier in the process - may be more amenable to some inspiration and simulation, as it is a simpler structure with clearer functions and pathways.  It handles some of the functions you described at a more basic level, including some edge detection, colour processing, contrast, and various other aspects of perception such as depth, brightness, and movement, albeit with many of the same caveats as mentioned for the visual cortex.</p>

CC BY-SA 3.0
2015-05-03T20:41:25.983Z
2015-05-04T03:36:14.477Z
2015-05-04T03:36:14.477Z
Arnon Weinberg
<p>Some time ago I remember reading about how the human brain breaks down visual information into a number of individual "channels".</p>

<p>For example, one channel might focus on edges and lines, another might focus on being a contrast map, while another provide a color information overlay. Each "channel" being a much more simplified part of the entire visual input.</p>

<p>From what I understood, modern research into the visual cortex has provided some insight into what the channels might be and how they're broken down. <em>If</em> my understanding of current research is correct I was hoping to find some more detailed information explaining how the visual information is broken down and processed (articles at the technical level of Scientific American are fine, so long as they detail the types of "channels" present).</p>

<p>My desired take-away from this is to brainstorm ideas for how I might better pre-process an image into feature data for a standard neural network learning algorithm, based on the human experience.</p>

<vision><neural-network>
How does the brain break down visual information for processing? What "channels" is visual input broken into?
David Parks
# How does the brain break down visual information for processing? What "channels" is visual input broken into?

(Asked by: username_0 on 2015-04-17)

Some time ago I remember reading about how the human brain breaks down visual information into a number of individual "channels".

For example, one channel might focus on edges and lines, another might focus on being a contrast map, while another provide a color information overlay. Each "channel" being a much more simplified part of the entire visual input.

From what I understood, modern research into the visual cortex has provided some insight into what the channels might be and how they're broken down. *If* my understanding of current research is correct I was hoping to find some more detailed information explaining how the visual information is broken down and processed (articles at the technical level of Scientific American are fine, so long as they detail the types of "channels" present).

My desired take-away from this is to brainstorm ideas for how I might better pre-process an image into feature data for a standard neural network learning algorithm, based on the human experience.

**Tags:** vision, neural-network


---

## 1 answers

### Answer by username_1 on 2015-05-03. Score: 8

The human visual processing system receives input from the eyes, and then passes it through a number of areas of the brain that break it down, process it in various different ways, recombine it, and break it down again several times. I'm assuming this question is only about the visual cortex, general theories about how information might be broken down for processing there, and only concerning parallel-processing (channels), as opposed to sequential specialization (layers), otherwise the answer should fill up a textbook.

**Main pathways through visual cortex:**

The [traditional understanding](http://en.wikipedia.org/wiki/Visual_cortex#Introduction) of the visual cortex divides it into [2 main pathways](http://en.wikipedia.org/wiki/Two-streams_hypothesis):

> Each V1 transmits information to two primary pathways, called the
> ventral stream and the dorsal stream.
>
> * The ventral stream begins with V1, goes through visual area V2, then through visual area V4, and to the inferior temporal cortex (IP
>   cortex). The ventral stream, sometimes called the "What Pathway", is
>   associated with form recognition and object representation. It is also
>   associated with storage of long-term memory.
> * The dorsal stream begins with V1, goes through Visual area V2, then to the dorsomedial area (DM/ V6) and Visual area MT (middle
>   temporal/ V5) and to the posterior parietal cortex. The dorsal stream,
>   sometimes called the "Where Pathway" or "How Pathway", is associated
>   with motion, representation of object locations, and control of the
>   eyes and arms, especially when visual information is used to guide
>   saccades or reaching.

The "what" and "where" pathways are also sometimes referred to as the "perception" and "action" pathways, respectively.

![enter image description here](https://i.sstatic.net/bBCa2.jpg)

Another [common theory](http://en.wikipedia.org/wiki/Visual_modularity) about the visual cortex identifies at least 3 modules: Motion, colour, and form. Evidence for these proposed modules comes largely from lesion studies and related disorders of the visual system. In general, an impairment in one module doesn't seem to affect the others, suggesting that these specializations are processed independently and in parallel. However, it is not clear if they form distinct pathways through the visual cortex.

There is also evidence for a [long-term memory pathway](http://onlinelibrary.wiley.com/doi/10.1002/hipo.20320/abstract;jsessionid=60AB8D41D856EA7A87DBEBEAEB81F9C9.f03t03) (visual-perirhinal-hippocampal stream) that feeds memory information from and to the visual cortex. Many other pathways have been postulated; this field of research is very active, consensus on further detail remains elusive for now.

**Complications to such models:**

Such "pathway" models of the visual cortex are limited by a number of factors:

* There is evidence of [significant interaction](http://www.sciencedirect.com/science/article/pii/S0959438898800421) between the streams
* There is evidence of [feedback cycles](http://www.ncbi.nlm.nih.gov/pubmed/20007766) within the streams
* Evidence that [different pathways](http://www.jneurosci.org/content/21/24/9844.short) are used in different situations
* There is some evidence of [asymmetric processing](http://www.ncbi.nlm.nih.gov/pubmed/8787209) in the brain
* There is also some evidence of [inter-person differences](http://www.ncbi.nlm.nih.gov/pubmed/17338604)

Furthermore, as mentioned, the visual cortex is only part of the visual system. Significant processing happens earlier - you mentioned about edge detection, some of that appears to already be happening by the time signals reach the visual cortex. And more processing happens later as well, in the parietal and temporal cortices for example, such as facial recognition.

**Implications for artificial neural networks:**

The visual cortex may not be ideal inspiration for ANNs, especially since it only handles a not-very-early part of the process, such that the signals it receives have already undergone significant modification from the raw stimuli. Additionally, its pathway structure is fairly complex, and not fully understood. There have nonetheless been [attempts to simulate](http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=4018419) parts of the visual cortex and compare to non-biologically inspired networks.

The [LGN](http://en.wikipedia.org/wiki/Lateral_geniculate_nucleus) - an area of the brain involved earlier in the process - may be more amenable to some inspiration and simulation, as it is a simpler structure with clearer functions and pathways. It handles some of the functions you described at a more basic level, including some edge detection, colour processing, contrast, and various other aspects of perception such as depth, brightness, and movement, albeit with many of the same caveats as mentioned for the visual cortex.


---

