<p>One of the fundamental operations in image neural networks is the operation of convolution, where a filter slides across the receptive field, activating on particular features. This is incredibly useful because it allows us to recycle filters, without having to train neurons on each piece of the receptive field. </p>

<p>It seems like such an operation is mechanically impossible <em>within</em> the human visual cortex. From the paper <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3306444/pdf/nihms352068.pdf" rel="nofollow noreferrer">How does the brain solve visual object recognition - DiCarlo, et. al.</a>, 
the brain opts to instead use a massive amount of neurons in each layer, most of which are redundant in that many neurons detect the same feature at different locations. This is clearly unscalable to apply everywhere, so only about 10% of the visual field concentrates these neurons. </p>

<p>On the other hand, we <em>do</em> have the ability to perform convolutions by moving our eyes. So in effect, we can jitter our eyes thereby applying the same neurons to different parts of the scene infront of us. I would think that this, combined with short-term memory can help us process images. </p>

<blockquote>
  <p>Suppose I force a subject to focus their eyes on the
  center of some common object (e.g. a car), such that the object is
  within the center 10% of our visual field. Are there situations where
  the brain is <em>unable</em> to process what the object is, <em>without</em>
  jittering our eyes? Said another way, how much visual processing are we able to do without any physical movement of our eyes or head?</p>
</blockquote>

<p>I'm interested in understanding this question because I'm not convinced that even the massive amount of neurons in the visual cortex are sufficient for visually parsing the object. Whereas by (even slightly) shifting the object in our vision, the combined featurizations from each location would significantly reduce the computational complexity.</p>

CC BY-SA 3.0
2018-04-30T20:05:52.633Z
The massively parallel convolution is only true for the simplest visual features which are important for detecting objects of all types (like edges). Cells with more specific best-stimuli always have bigger receptive fields.
Bryan Krause
CC BY-SA 3.0
2018-04-30T21:22:01.123Z
@BryanKrause: But I think the "bigger receptive field" here is still around 10 degrees of vision (for example see Figure 4C on page 33 of the above link). That particular IT neuron is responding to binoculars and an otter, but not a face. It's still not clear what features exactly that neuron is capturing, but they are surely higher order combinations of lower-level features.
Alex R.
CC BY-SA 4.0
2020-07-28T23:05:34.597Z
This is a really good question, that I wish would receive a full answer. (Came here to ask basically the same thing.)  Introductions to convolutional (artificial) neural networks seem to always gloss over the fact that the weights are *shared* across the visual field in the computation, but that in the biologically-inspired structure (cat visual cortexes)  I can't see any biological way for the neurons in different parts of the visual field to *share weights* the way we do computationally, apart from maybe @AlexR.'s notion of "jitter".
sh37211
CC BY-SA 4.0
2020-07-28T23:28:25.393Z
As far as I can tell, convolution was introduced by Fukushima in 1980 *not* because it was biologically-inspired, but rather because it allowed the artificial network to be less sensitive to changes in position.  (His biologically-inspired part was the hierarchical arrangement of the layers.)  ...but I'm kind of "asking" here, not "telling"! ;-)
sh37211
<cognitive-neuroscience><vision><neural-network>
Convolutional Operations in the Visual Cortex
Alex R.
# Convolutional Operations in the Visual Cortex

(Asked by: username_0 on 2018-04-30)

One of the fundamental operations in image neural networks is the operation of convolution, where a filter slides across the receptive field, activating on particular features. This is incredibly useful because it allows us to recycle filters, without having to train neurons on each piece of the receptive field.

It seems like such an operation is mechanically impossible *within* the human visual cortex. From the paper [How does the brain solve visual object recognition - DiCarlo, et. al.](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3306444/pdf/nihms352068.pdf),
the brain opts to instead use a massive amount of neurons in each layer, most of which are redundant in that many neurons detect the same feature at different locations. This is clearly unscalable to apply everywhere, so only about 10% of the visual field concentrates these neurons.

On the other hand, we *do* have the ability to perform convolutions by moving our eyes. So in effect, we can jitter our eyes thereby applying the same neurons to different parts of the scene infront of us. I would think that this, combined with short-term memory can help us process images.

> Suppose I force a subject to focus their eyes on the
> center of some common object (e.g. a car), such that the object is
> within the center 10% of our visual field. Are there situations where
> the brain is *unable* to process what the object is, *without*
> jittering our eyes? Said another way, how much visual processing are we able to do without any physical movement of our eyes or head?

I'm interested in understanding this question because I'm not convinced that even the massive amount of neurons in the visual cortex are sufficient for visually parsing the object. Whereas by (even slightly) shifting the object in our vision, the combined featurizations from each location would significantly reduce the computational complexity.

**Tags:** cognitive-neuroscience, vision, neural-network

> **Comments:**
> **username_1**: The massively parallel convolution is only true for the simplest visual features which are important for detecting objects of all types (like edges). Cells with more specific best-stimuli always have bigger receptive fields.  
> **username_0**: @BryanKrause: But I think the "bigger receptive field" here is still around 10 degrees of vision (for example see Figure 4C on page 33 of the above link). That particular IT neuron is responding to binoculars and an otter, but not a face. It's still not clear what features exactly that neuron is capturing, but they are surely higher order combinations of lower-level features.  
> **username_2**: This is a really good question, that I wish would receive a full answer. (Came here to ask basically the same thing.)  Introductions to convolutional (artificial) neural networks seem to always gloss over the fact that the weights are *shared* across the visual field in the computation, but that in the biologically-inspired structure (cat visual cortexes)  I can't see any biological way for the neurons in different parts of the visual field to *share weights* the way we do computationally, apart from maybe @AlexR.'s notion of "jitter".  
> **username_2**: As far as I can tell, convolution was introduced by Fukushima in 1980 *not* because it was biologically-inspired, but rather because it allowed the artificial network to be less sensitive to changes in position.  (His biologically-inspired part was the hierarchical arrangement of the layers.)  ...but I'm kind of "asking" here, not "telling"! ;-)  

---

## 0 answers

