<p>Treisman &amp; Gelade's <a href="http://en.wikipedia.org/wiki/Feature_integration_theory" rel="noreferrer">Feature Integration Theory</a> suggests that we are able to process an entire visual scene in parallel at the level of individual features. For example, in a visual search task, the time required to find a blue circle in a field of red circles is <em>independent of the total number of circles</em>. However, focused attention (typically foveal) is required to integrate independent features into a cohesive object. Thus, if searching for a red circle in a field of blue circles and red squares, search time grows linearly with the number of total objects. This is because the target is made up of two features (circle and red) which need to be integrated in order to be identified-- requiring saccades around the scene.</p>
<p>Several theories of visual search use this distinction to model visual attention shifts. Most notably, Jeremy Wolfe's Guided Search and Itti &amp; Koch's visual attention model. The basic premise behind both models is somewhat similar: low level feature receptors respond automatically and in parallel to the entire visual field. Thus, there are many individual feature maps that represent <em>bottom-up saliency</em> of locations in the visual scene. This bottom-up saliency can be sufficient to trigger a saccade; for instance, a feature map that responds to local motion is beneficial for an organism to identify moving predators. Thus, areas with motion are given high value because they have a history of providing information that is beneficial to an organism.</p>
<p>During task conditions (such as visual search), <em>top-down saliency</em> maps may also be created based on knowledge of what things in the environment have value. If I am searching for my umbrella, I know that it is blue and long and straight, and this information can be encoded in the feature maps that drive saccades.</p>
<p>More generally, saccades are directed at targets that have a high expected value. (It has even been shown that the velocity of saccades is proportional to the expected value of the target: Shadmehr, et al.) This value is determined from a weighted evaluation of both top-down and bottom-up feature maps, available pre-attentively.</p>
<p>The <em>exact</em> location of a saccade is determined through a process called spatial pooling, which attempts to determine the 'center of gravity' of a target, again using low level feature maps. While saccades are amazingly quick and accurate, there is of course some error in final saccadic position which often require smaller saccades to reach the target. It has recently been suggested that this series of saccadic movements emulate Fitts' law with regards to speed-accuracy tradeoffs. A great, thorough review of the current state of saccadic eye movements can be found in Kowler, 2011.</p>
<p>There is obviously quite a bit of detailed information that I haven't covered here-- out of the references cited, I would start with section 3 (&quot;Saccades&quot;) of the Kowler article, then move on to the Itti &amp; Koch article for more concrete details on their specific model.</p>
<blockquote>
<p>Wolfe, J.M. (1994). Guided Search 2.0. A revised model of visual search. Psychonomic Bulletin &amp; Review, 1, 202-238.</p>
<p>Itti, L., &amp; Koch, C. (2001). Computational modelling of visual
attention. Nature Reviews Neuroscience, 2, 1-11.</p>
<p>Shadmehr et al. (2010). Temporal discounting of reward and the cost of
time in motor control. Journal of Neuroscience, 30, 10507-10516.</p>
<p>Kowler, E. (2011). Eye movements: The past 25 years. Vision Research,
51, 1457-1483.</p>
</blockquote>
<p>** Note I cite Guided Search 2.0 because it is a good exposition of the whole theory, though the theory itself has progressed to 4.0</p>

CC BY-SA 3.0
2012-07-08T17:04:11.240Z
You may also want to check out this paper and the models discussed therein. SWIFT and EZ-Reader are both models of oculomotor control in reading. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2906818/
Pavel
CC BY-SA 3.0
2012-02-20T06:05:18.643Z
2012-02-20T14:20:55.200Z
2020-06-17T09:55:31.633Z
Jeff
<p>I've gathered the standard rational for a visual system utilizing saccades from perception textbooks: the neural cost of processing an entire scene at a high level of detail would be prohibitive, but low-fidelity images aren't good enough to function in the world. Thus you have an retina with a high-fidelity center which can sample the scene by <a href="http://en.wikipedia.org/wiki/Saccade">saccading</a> around, presumably guided by a combination of information about the low-fidelity surround and the beliefs and goals of the perceiver.</p>

<p>However, I've yet to find a detailed theory that answers the question of <strong>how the next saccade target selected.</strong></p>

<p>To clarify, by 'detailed theory', I mean a theory that describes the <strong>mechanisms</strong> used to accomplish the computational feats described in the standard textbook explanation I mentioned above.</p>

<computational-modeling><perception><vision><eye-movement>
How do humans control saccades?
zergylord
# How do humans control saccades?

(Asked by: username_0 on 2012-02-18)

I've gathered the standard rational for a visual system utilizing saccades from perception textbooks: the neural cost of processing an entire scene at a high level of detail would be prohibitive, but low-fidelity images aren't good enough to function in the world. Thus you have an retina with a high-fidelity center which can sample the scene by [saccading](http://en.wikipedia.org/wiki/Saccade) around, presumably guided by a combination of information about the low-fidelity surround and the beliefs and goals of the perceiver.

However, I've yet to find a detailed theory that answers the question of **how the next saccade target selected.**

To clarify, by 'detailed theory', I mean a theory that describes the **mechanisms** used to accomplish the computational feats described in the standard textbook explanation I mentioned above.

**Tags:** computational-modeling, perception, vision, eye-movement


---

## 1 answers

### Answer by username_1 on 2012-02-20. Score: 16 | Accepted answer

Treisman & Gelade's [Feature Integration Theory](http://en.wikipedia.org/wiki/Feature_integration_theory) suggests that we are able to process an entire visual scene in parallel at the level of individual features. For example, in a visual search task, the time required to find a blue circle in a field of red circles is *independent of the total number of circles*. However, focused attention (typically foveal) is required to integrate independent features into a cohesive object. Thus, if searching for a red circle in a field of blue circles and red squares, search time grows linearly with the number of total objects. This is because the target is made up of two features (circle and red) which need to be integrated in order to be identified-- requiring saccades around the scene.

Several theories of visual search use this distinction to model visual attention shifts. Most notably, Jeremy Wolfe's Guided Search and Itti & Koch's visual attention model. The basic premise behind both models is somewhat similar: low level feature receptors respond automatically and in parallel to the entire visual field. Thus, there are many individual feature maps that represent *bottom-up saliency* of locations in the visual scene. This bottom-up saliency can be sufficient to trigger a saccade; for instance, a feature map that responds to local motion is beneficial for an organism to identify moving predators. Thus, areas with motion are given high value because they have a history of providing information that is beneficial to an organism.

During task conditions (such as visual search), *top-down saliency* maps may also be created based on knowledge of what things in the environment have value. If I am searching for my umbrella, I know that it is blue and long and straight, and this information can be encoded in the feature maps that drive saccades.

More generally, saccades are directed at targets that have a high expected value. (It has even been shown that the velocity of saccades is proportional to the expected value of the target: Shadmehr, et al.) This value is determined from a weighted evaluation of both top-down and bottom-up feature maps, available pre-attentively.

The *exact* location of a saccade is determined through a process called spatial pooling, which attempts to determine the 'center of gravity' of a target, again using low level feature maps. While saccades are amazingly quick and accurate, there is of course some error in final saccadic position which often require smaller saccades to reach the target. It has recently been suggested that this series of saccadic movements emulate Fitts' law with regards to speed-accuracy tradeoffs. A great, thorough review of the current state of saccadic eye movements can be found in Kowler, 2011.

There is obviously quite a bit of detailed information that I haven't covered here-- out of the references cited, I would start with section 3 ("Saccades") of the Kowler article, then move on to the Itti & Koch article for more concrete details on their specific model.

> Wolfe, J.M. (1994). Guided Search 2.0. A revised model of visual search. Psychonomic Bulletin & Review, 1, 202-238.
>
> Itti, L., & Koch, C. (2001). Computational modelling of visual
> attention. Nature Reviews Neuroscience, 2, 1-11.
>
> Shadmehr et al. (2010). Temporal discounting of reward and the cost of
> time in motor control. Journal of Neuroscience, 30, 10507-10516.
>
> Kowler, E. (2011). Eye movements: The past 25 years. Vision Research,
> 51, 1457-1483.

\*\* Note I cite Guided Search 2.0 because it is a good exposition of the whole theory, though the theory itself has progressed to 4.0

> **Comments:**
> **username_2**: You may also want to check out this paper and the models discussed therein. SWIFT and EZ-Reader are both models of oculomotor control in reading. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2906818/  

---

