<p>I don't buy it the claim made in the quote. The speed of sound is roughly 1 foot per millsecond so even if you take a large 3 foot step you are changing the audio visual onset asynchrony by only 3 ms. What this means is that if you present a flash and a click with various onset asynchrony to subjects you would expect there to be a narrow range of lags (less than 3 ms) over which they stop reporting that the flash occurred first. This study was run by <a href="http://download.springer.com/static/pdf/285/art%253A10.3758%252FPP.70.6.955.pdf?auth66=1411724940_c6ca400a0ac5f5955debf70b4619f206&amp;ext=.pdf" rel="nofollow noreferrer">van Eijk et al (2008)</a>: <img src="https://i.sstatic.net/KkDKI.png" alt="enter image description here"></p>

<p>The relevant data are the temporal order judgements (TOJ) and it is clear that it is a smooth transition between thinking the flash was first and the click was first and not an abrupt step. Further, the transition takes around 200 ms or to get back to the original claim, about 100 steps. The 80 ms claim seems reasonable in that you do not consistently report the visual stimulus as occurring first more than 70% of the time until it occurs more than 80 ms before the sound.</p>

CC BY-SA 3.0
2014-09-25T12:37:36.733Z
this is very interesting and I'll give it an upvote for adding value, but it doesn't directly answer the question. I have summarized my question, in the original post, to add some clarity.
Jordan
CC BY-SA 3.0
2014-09-24T10:05:40.800Z
2014-09-24T10:05:40.800Z
StrongBad
<p>The answer is definitely yes, if you take a slightly different example <a href="http://link.springer.com/article/10.3758/BF03206096#" rel="nofollow">MacDonald &amp; McGurk (1978)</a>.  The McGurk effect in linguistics is quite well-known: given video of a mouth pronouncing a bilabial consonant, and synced audio of a nonlabial consonant, the viewer will generally report hearing a consonant whose place of articulation is roughly the average of the bilabial vs nonlabial POAs.  (The audio of the nonlabial consonant alone is generally identified accurately.)  Speech is often a bit of a special case when it comes to perception, though.</p>

<p>MacDonald, J., &amp; McGurk, H. (1978). Visual influences on speech perception processes, <em>Perception &amp; Psychophysics</em> May 1978, Volume 24, Issue 3, pp 253-257. DOI 10.3758/BF03206096</p>

CC BY-SA 3.0
2014-09-22T18:16:57.473Z
But isn't this a somewhat different mechanism? Isn't this more along the lines of - if you were to synchronize a speaker's lips with a sound such that the speaker would say, without actually making a sound, "bar" and the audio would say "var" or "far" - the listener, who would be told to watch the speakers lips while the sound plays, would still think they heard "bar"?
Jordan
CC BY-SA 3.0
2014-09-22T18:19:18.087Z
It probably is at least partially a different mechanism, since both space and language have their own interplays with vision; but it is an example of auditory perception warping visual perception (since people don't perceive the clear visual cues of bilabial production when paired with non-labial sound).
Krysta
CC BY-SA 3.0
2014-09-22T18:31:15.917Z
Well yeah they are related, but I feel like the McGurk effect is related more to our perception of language where this would be the perception of depth. I feel like maybe this would be somewhat closer: http://scienceblogs.com/cognitivedaily/2007/01/09/what-we-hear-and-how-it-affect/ but even this has to do with the perception of motion and it's unclear, to me, how depth would play into that. Short of running an experiment, I imagine our visual perception could remain dominant over our audible perception and therefore we may not see a change in depth perception.
Jordan
CC BY-SA 3.0
2014-09-22T18:31:35.843Z
I guess I could write a program to test this if I find the time at some point..
Jordan
CC BY-SA 3.0
2014-09-22T12:26:29.400Z
2014-09-22T12:26:29.400Z
Krysta
<p>"Our consciousness lags 80 milliseconds behind actual events...The 80-millisecond rule plays all sorts of perceptual tricks on us. As long as a hand-clapper is less than 30 meters away, you hear and see the clap happen together. But beyond this distance, the sound arrives more than 80 milliseconds later than the light, and the brain no longer matches sight and sound. What is weird is that the transition is abrupt: by taking a single step away from you, the hand-clapper goes from in sync to out of sync." <a href="http://blogs.scientificamerican.com/observations/2011/09/15/time-on-the-brain-how-you-are-always-living-in-the-past-and-other-quirks-of-perception/" rel="nofollow">source</a></p>

<p>This made me curious - say we have the following experiment based on the above:</p>

<ul>
<li>A <strong>subject (listening subject)</strong> wears headphones synchronized with the <strong>hand-clapper</strong></li>
<li>the <strong>hand-clapper</strong> starts from 5 meters away and slowly moves to 60 meters away
<ul>
<li>during this time, the <strong>listening subject</strong> would hear a clapping sound as they would naturally hear the sound <em>based on</em> the distance of the <strong>hand-clapper</strong></li>
</ul></li>
</ul>

<p>If, once the <strong>hand-clapper</strong> reaches 60 meters, the audio is <em>tweaked</em> such that the <strong>listening subject</strong> would hear the hand-clapping sound <em>as if</em> the <strong>hand-clapper</strong> were only 5 meters away: </p>

<blockquote>
  <p>Would your typical <strong>listening subject</strong> visually perceive the
  <strong>hand-clapper</strong> as having suddenly moved closer (similar to the concept of how we perceive the moon as being "larger" when it's closer
  to the horizon)?</p>
  
  <p>In other words -- under what circumstances, if any, can sound influence
  our visual perception of depth/proximity?</p>
</blockquote>

<perception><cognitive-neuroscience>
Can our auditory perception be used to trick our visual perception?
Jordan
# Can our auditory perception be used to trick our visual perception?

(Asked by: username_0 on 2014-09-20)

"Our consciousness lags 80 milliseconds behind actual events...The 80-millisecond rule plays all sorts of perceptual tricks on us. As long as a hand-clapper is less than 30 meters away, you hear and see the clap happen together. But beyond this distance, the sound arrives more than 80 milliseconds later than the light, and the brain no longer matches sight and sound. What is weird is that the transition is abrupt: by taking a single step away from you, the hand-clapper goes from in sync to out of sync." [source](http://blogs.scientificamerican.com/observations/2011/09/15/time-on-the-brain-how-you-are-always-living-in-the-past-and-other-quirks-of-perception/)

This made me curious - say we have the following experiment based on the above:

* A **subject (listening subject)** wears headphones synchronized with the **hand-clapper**
* the **hand-clapper** starts from 5 meters away and slowly moves to 60 meters away
  + during this time, the **listening subject** would hear a clapping sound as they would naturally hear the sound *based on* the distance of the **hand-clapper**

If, once the **hand-clapper** reaches 60 meters, the audio is *tweaked* such that the **listening subject** would hear the hand-clapping sound *as if* the **hand-clapper** were only 5 meters away:

> Would your typical **listening subject** visually perceive the
> **hand-clapper** as having suddenly moved closer (similar to the concept of how we perceive the moon as being "larger" when it's closer
> to the horizon)?
>
> In other words -- under what circumstances, if any, can sound influence
> our visual perception of depth/proximity?

**Tags:** perception, cognitive-neuroscience


---

## 2 answers

### Answer by username_1 on 2014-09-24. Score: 3

I don't buy it the claim made in the quote. The speed of sound is roughly 1 foot per millsecond so even if you take a large 3 foot step you are changing the audio visual onset asynchrony by only 3 ms. What this means is that if you present a flash and a click with various onset asynchrony to subjects you would expect there to be a narrow range of lags (less than 3 ms) over which they stop reporting that the flash occurred first. This study was run by [van Eijk et al (2008)](http://download.springer.com/static/pdf/285/art%253A10.3758%252FPP.70.6.955.pdf?auth66=1411724940_c6ca400a0ac5f5955debf70b4619f206&ext=.pdf): ![enter image description here](https://i.sstatic.net/KkDKI.png)

The relevant data are the temporal order judgements (TOJ) and it is clear that it is a smooth transition between thinking the flash was first and the click was first and not an abrupt step. Further, the transition takes around 200 ms or to get back to the original claim, about 100 steps. The 80 ms claim seems reasonable in that you do not consistently report the visual stimulus as occurring first more than 70% of the time until it occurs more than 80 ms before the sound.

> **Comments:**
> **username_0**: this is very interesting and I'll give it an upvote for adding value, but it doesn't directly answer the question. I have summarized my question, in the original post, to add some clarity.  

---

### Answer by username_2 on 2014-09-22. Score: 2

The answer is definitely yes, if you take a slightly different example [MacDonald & McGurk (1978)](http://link.springer.com/article/10.3758/BF03206096#). The McGurk effect in linguistics is quite well-known: given video of a mouth pronouncing a bilabial consonant, and synced audio of a nonlabial consonant, the viewer will generally report hearing a consonant whose place of articulation is roughly the average of the bilabial vs nonlabial POAs. (The audio of the nonlabial consonant alone is generally identified accurately.) Speech is often a bit of a special case when it comes to perception, though.

MacDonald, J., & McGurk, H. (1978). Visual influences on speech perception processes, *Perception & Psychophysics* May 1978, Volume 24, Issue 3, pp 253-257. DOI 10.3758/BF03206096

> **Comments:**
> **username_0**: But isn't this a somewhat different mechanism? Isn't this more along the lines of - if you were to synchronize a speaker's lips with a sound such that the speaker would say, without actually making a sound, "bar" and the audio would say "var" or "far" - the listener, who would be told to watch the speakers lips while the sound plays, would still think they heard "bar"?  
> **username_2**: It probably is at least partially a different mechanism, since both space and language have their own interplays with vision; but it is an example of auditory perception warping visual perception (since people don't perceive the clear visual cues of bilabial production when paired with non-labial sound).  
> **username_0**: Well yeah they are related, but I feel like the McGurk effect is related more to our perception of language where this would be the perception of depth. I feel like maybe this would be somewhat closer: http://scienceblogs.com/cognitivedaily/2007/01/09/what-we-hear-and-how-it-affect/ but even this has to do with the perception of motion and it's unclear, to me, how depth would play into that. Short of running an experiment, I imagine our visual perception could remain dominant over our audible perception and therefore we may not see a change in depth perception.  
> **username_0**: I guess I could write a program to test this if I find the time at some point..  

---

