<p>Chris gave an excellent answer for visual-to-tactile sensory substitution devices. I will not argue about the extent to which sensory substitution is successful, but the most widespread sensory substitution system for the blind is nowadays <a href="http://www.seeingwithsound.com" rel="nofollow">The vOICe</a>, a class of visual-to-auditory sensory substitution devices that is available globally. Its Android version has been installed over a quarter million times in the last few years, and there are a number of recent peer-reviewed <a href="http://www.seeingwithsound.com/literature.htm" rel="nofollow">scientific publications</a> based on use of its Microsoft Windows version, covering various aspects of brain plasticity. Upcoming augmented reality glasses are likely to improve convenience of use. But yes, like Chris stated, learning to use a SSD effectively takes a lot of time.</p>

CC BY-SA 3.0
2015-01-24T10:15:38.723Z
Nice answer! +1 are you related to The vOICe? I focused my question pretty much on Bach-y-Rita's legacy given the question. I think the main downsides of BrainPort, except for discreteness, also apply to other SSDs.
AliceD
CC BY-SA 3.0
2015-01-24T14:32:09.780Z
Thanks @Chris. Yes I'm related to The vOICe so I could be biased. However, The vOICe is far cheaper than the BrainPort device, offers a much higher resolution (e.g. 176 x 64 versus only 12 x 12 for the 144-electrode BrainPort), and it works with mass-market hardware. Oh, and you can speak and eat while using it. :-) On the other hand, the BrainPort should work better for people who are deaf-blind, and it may work better for tracking motion.
seeingwithsound
CC BY-SA 3.0
2015-01-25T00:50:25.043Z
Will you be going to arvo :)
AliceD
CC BY-SA 3.0
2015-01-25T19:42:07.857Z
@Chris You asked the same question by e-mail last year. :-) Not this year, but there was an invited presentation on The vOICe at [Asia-ARVO 2013](http://www.seeingwithsound.com/extra/AsiaARVO2013_MeijerPBL.pdf) in New Delhi, India (PDF of slides).
seeingwithsound
CC BY-SA 3.0
2015-02-06T13:12:41.687Z
thanks for the slides!! very helpful. Could you please provide a reference to the resolution you mention? I am writing this article and I can't find a decent source. Thanks.
AliceD
CC BY-SA 3.0
2015-02-06T17:57:05.880Z
The 176 x 64 hardware/software resolution is mentioned on the product pages for the [Microsoft Windows](http://www.seeingwithsound.com/winvoice.htm) and [Android](http://www.seeingwithsound.com/android.htm) versions of The vOICe. Perceptual resolution with The vOICe in the form of "visual" acuity is discussed in the peer-reviewed [PLOS ONE](http://dx.doi.org/10.1371/journal.pone.0033136) and [Frontiers in Psychology](http://dx.doi.org/10.3389/fpsyg.2013.00330) papers.
seeingwithsound
CC BY-SA 3.0
2015-01-23T22:34:26.603Z
2015-01-23T22:34:26.603Z
seeingwithsound
<p><a href="http://en.m.wikipedia.org/wiki/Tadoma" rel="nofollow">Tadoma</a> is a method of communicating used by some deaf blind individuals. The system was developed prior to 1967 and predates TVSS, although there is no hardware involved. I think this <a href="https://m.youtube.com/watch?v=U_QMS-hzRMs" rel="nofollow">video</a> demonstrates the extent to which the approach is successful, and hence not limited by brain plasticity. Tadoma has fallen out of favor because it is difficult to learn and teach. There has been some research into vibrotactile hearing aids and there are some <a href="http://deafness.about.com/cs/vibrotactile/f/vibrotactile.htm" rel="nofollow">commercially available</a>.</p>

CC BY-SA 3.0
2015-02-06T13:28:17.887Z
Tadoma is a non-electronic way of sensory substitution. Guide dogs and the guide cane pre-date the TVSS as well, obviously. The link http://deafness.about.com/cs/vibrotactile/f/vibrotactile.htm is quite obscure, as it links to dead websites, and weird unintelligible Chinese matter, unfortunately.
AliceD
CC BY-SA 3.0
2015-01-23T22:47:01.300Z
2015-01-23T22:47:01.300Z
StrongBad
<p>Bach-y-Rita's Tactile Vision Substitution System (TVSS) project was initiated in 1963 and he has since been regarded as the founding father of <strong>sensory substitution</strong>. The concept of sensory substitution refers to the process of obtaining information about the world from a functional sensory system (e.g. touch) that would normally be obtained from a lost sensory system (e.g, vision in the blind). </p>

<p>Bach-y-Rita developed various devices, but his most cited one consisted of 400 vibrotactile solenoids mounted in the back of a dental chair. Camera images were converted into binary ("black and white") tactile representations onto the back of a subject ('seeing with the 'skin') (<a href="http://www.nature.com/nature/journal/v221/n5184/abs/221963a0.html" rel="nofollow noreferrer">Bach-y-Rita 1969</a>):</p>

<p><img src="https://i.sstatic.net/fnJ33.jpg" alt="TVSS"></p>

<p>The results obtained with the TVSS were very promising: after sufficient practice (20 – 150 hours), blind subjects were able to perceive movement, recognize shapes and objects, and even identify faces. It also enabled the appreciation of more abstract concepts quite unfamiliar to the blind, such as shadow, depth, looming, zooming and parallax (<a href="http://www.tandfonline.com/doi/abs/10.1207/S15327590IJHC1502_6?journalCode=hihc20" rel="nofollow noreferrer">Bach-y-Rita, 2003)</a>. A face projected onto the skin would have felt like something like this (photo of an oscilloscope image representing the voltage stimuli sent from the camera to the vibrotactile display).</p>

<p><img src="https://i.sstatic.net/K7PqL.jpg" alt="TVSS binary"></p>

<p>However, as the first figure shows, the TVSS was large and bulky. Moreover, it was noisy and power hungry and simply not very useful outside the laboratory. Although various portable variants were constructed, they too suffered these limitations in various degrees:</p>

<p><img src="https://i.sstatic.net/7PCzr.jpg" alt="PortableTVSS"></p>

<p>Bach-y-Rita's work has been extended into <em>electrotactile</em> approaches and the company Wicab is currently marketing the BrainPort V100 device in Europe. It features a head-worn camera to scan the environment and translates these images onto a grid of electrodes that stimulate the tongue to elicit electrotactile sensations (much like a sip of champagne or a soda would do):</p>

<p><img src="https://i.sstatic.net/KxxDu.jpg" alt="BrainPort"></p>

<p>It uses electrodes instead of bulky vibrotactors which greatly reduces power consumption and this approach has made a portable application feasible. Electrodes, however, need a moist surface or very high voltages are necessary. A straightforward approach was to apply the electrode grid onto the tongue. The tongue is moist and has excellent resolution. The BrainPort currently features 400 electrodes and is able to provide graded voltages to present gray scale images onto the tongue. </p>

<p>The BrainPort has been marketed fairly recently in Europe (2013) and an FDA study in the US has just been finished. As far as I know they are busy with commercializing there as well.</p>

<p>Regarding your <strong>brain plasticity</strong> question: There is a wealth of information available on neuroplastic changes in the brain due to the use of the BrainPort or other sensory substitution approaches (VoiCE, EyeMusic etc etc). Basically it has been shown that the deafferented visual cortices are recruited for tactile information processing (e.g. <a href="http://www.ncbi.nlm.nih.gov/pubmed/12030824" rel="nofollow noreferrer">Sadato, 2002</a>), and hence, the brain actively adapts to the new source of information. So <strong>no</strong>, plastic changes are <strong>not</strong> the limiting factor.</p>

<p>Hence, the BrainPort (and other sensory substitution approaches like the SonicGuide etc.) is useful, portable, and not less importantly, the brain <em>actively adapts</em> in using the alternative visuo-tactile input by recruiting the visual cortex for processing visuotactile information. <strong>Why then haven't sensory substitution approaches conquered the world of visual rehabilitation of the blind</strong>"? There are a number of important downsides to sensory substitution devices (SSDs):</p>

<ul>
<li>SSDs <strong>substitute</strong> vision (by tactile or acoustic information), they do not <strong>restore</strong> it;</li>
<li>The <strong>limited resolution</strong> of SSDs does not allow for detailed target analysis, i.e., discerning the difference between a fork and a knife will be very difficult;</li>
<li>Instead, <strong>relying on well-established coping mechanisms</strong> and sticking to their usual routines may be far more effective in everyday life. For example, simply touching the fork or spoon will immediately tell them the difference, while scanning them with a SSD takes them ages (i.e., SSDs are not very practical);</li>
<li>The restricted resolution of state-of-the-art SSDs also aids little in orientation and mobility (O&amp;M). In other words, it does not help blind folks to go about. Instead, <strong>established methods such as the guide cane and guide dog are far more effective for O&amp;M</strong>;</li>
<li>Learning to use a SSD effectively takes <em>a lot of time</em>. <strong>Not many people are willing to undergo an arduous rehabilitation program</strong>;</li>
<li>In case of the BrainPort - it is very visible to the outside world and <strong>not very discreet</strong> when a blind person is using it. It is known that 29% of consumers with disabilities abandon the use of assistive devices for reasons unrelated to device performance (<a href="https://www.google.com.au/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;cad=rja&amp;uact=8&amp;ved=0CCIQFjAA&amp;url=http%3A%2F%2Fcs.brynmawr.edu%2FCourses%2Fcs380%2Ffall2010%2Freadings%2FPhillips1993.pdf&amp;ei=a73BVNysIM7n8AWqmoKACQ&amp;usg=AFQjCNEeto4iQFmsrJQl_Gsz6XouCUuAAA&amp;sig2=5qx3mDY7gEuhr20TvpnW0w&amp;bvm=bv.84349003,d.dGc" rel="nofollow noreferrer">Phillips and Zhao, 1993</a>).  </li>
</ul>

<p><sup><strong>References</strong></sup><br>
<sub><strong>-</strong> <a href="http://www.nature.com/nature/journal/v221/n5184/abs/221963a0.html" rel="nofollow noreferrer">Bach-y-Rita 1969. <em>Nature</em> 221:963-4</a></sub><br>
<sub><strong>-</strong> <a href="https://i.sstatic.net/fnJ33.jpg" rel="nofollow noreferrer">Bach-y-Rita 2003. <em>JHCI</em> <strong>15</strong>:285-95</a></sub><br>
<sub><strong>-</strong> <a href="https://www.google.com.au/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;cad=rja&amp;uact=8&amp;ved=0CCIQFjAA&amp;url=http%3A%2F%2Fcs.brynmawr.edu%2FCourses%2Fcs380%2Ffall2010%2Freadings%2FPhillips1993.pdf&amp;ei=a73BVNysIM7n8AWqmoKACQ&amp;usg=AFQjCNEeto4iQFmsrJQl_Gsz6XouCUuAAA&amp;sig2=5qx3mDY7gEuhr20TvpnW0w&amp;bvm=bv.84349003,d.dGc" rel="nofollow noreferrer">Philips &amp; Zhao 1993. <em>Assistive technology</em> <strong>5</strong>:36-45</a></sub><br>
<sub><strong>-</strong> <a href="http://www.ncbi.nlm.nih.gov/pubmed/12030824" rel="nofollow noreferrer">Sadato 2002. <em>Neuroimage</em> <strong>16</strong>:389-400</a></sub>  </p>

CC BY-SA 3.0
2015-01-23T06:15:59.870Z
wow thanks Chris for this extensive overview of the current state of such devices!
Mehdi
CC BY-SA 4.0
2015-01-23T03:19:56.210Z
2018-09-07T20:30:37.660Z
2018-09-07T20:30:37.660Z
AliceD
<p>By successful, I mean made it out to the large world market or being massively funded. After looking online it seems that the most relevant research is the one done by Bach-y-Rita about the <a href="http://www.youtube.com/watch?v=OKd56D2mvN0" rel="nofollow noreferrer">seeing with tongue</a> in the 70's. Since then there were no significant breakthroughs. Is there a reason for that? Did someone prove that brain plasticity is limited?</p>

<neuroscience><perception><vision><sensation><plasticity>
Why is sensory substitution not that successful?
Mehdi
# Why is sensory substitution not that successful?

(Asked by: username_0 on 2015-01-23)

By successful, I mean made it out to the large world market or being massively funded. After looking online it seems that the most relevant research is the one done by Bach-y-Rita about the [seeing with tongue](http://www.youtube.com/watch?v=OKd56D2mvN0) in the 70's. Since then there were no significant breakthroughs. Is there a reason for that? Did someone prove that brain plasticity is limited?

**Tags:** neuroscience, perception, vision, sensation, plasticity


---

## 3 answers

### Answer by username_2 on 2015-01-23. Score: 10 | Accepted answer

Bach-y-Rita's Tactile Vision Substitution System (TVSS) project was initiated in 1963 and he has since been regarded as the founding father of **sensory substitution**. The concept of sensory substitution refers to the process of obtaining information about the world from a functional sensory system (e.g. touch) that would normally be obtained from a lost sensory system (e.g, vision in the blind).

Bach-y-Rita developed various devices, but his most cited one consisted of 400 vibrotactile solenoids mounted in the back of a dental chair. Camera images were converted into binary ("black and white") tactile representations onto the back of a subject ('seeing with the 'skin') ([Bach-y-Rita 1969](http://www.nature.com/nature/journal/v221/n5184/abs/221963a0.html)):

![TVSS](https://i.sstatic.net/fnJ33.jpg)

The results obtained with the TVSS were very promising: after sufficient practice (20 – 150 hours), blind subjects were able to perceive movement, recognize shapes and objects, and even identify faces. It also enabled the appreciation of more abstract concepts quite unfamiliar to the blind, such as shadow, depth, looming, zooming and parallax ([Bach-y-Rita, 2003)](http://www.tandfonline.com/doi/abs/10.1207/S15327590IJHC1502_6?journalCode=hihc20). A face projected onto the skin would have felt like something like this (photo of an oscilloscope image representing the voltage stimuli sent from the camera to the vibrotactile display).

![TVSS binary](https://i.sstatic.net/K7PqL.jpg)

However, as the first figure shows, the TVSS was large and bulky. Moreover, it was noisy and power hungry and simply not very useful outside the laboratory. Although various portable variants were constructed, they too suffered these limitations in various degrees:

![PortableTVSS](https://i.sstatic.net/7PCzr.jpg)

Bach-y-Rita's work has been extended into *electrotactile* approaches and the company Wicab is currently marketing the BrainPort V100 device in Europe. It features a head-worn camera to scan the environment and translates these images onto a grid of electrodes that stimulate the tongue to elicit electrotactile sensations (much like a sip of champagne or a soda would do):

![BrainPort](https://i.sstatic.net/KxxDu.jpg)

It uses electrodes instead of bulky vibrotactors which greatly reduces power consumption and this approach has made a portable application feasible. Electrodes, however, need a moist surface or very high voltages are necessary. A straightforward approach was to apply the electrode grid onto the tongue. The tongue is moist and has excellent resolution. The BrainPort currently features 400 electrodes and is able to provide graded voltages to present gray scale images onto the tongue.

The BrainPort has been marketed fairly recently in Europe (2013) and an FDA study in the US has just been finished. As far as I know they are busy with commercializing there as well.

Regarding your **brain plasticity** question: There is a wealth of information available on neuroplastic changes in the brain due to the use of the BrainPort or other sensory substitution approaches (VoiCE, EyeMusic etc etc). Basically it has been shown that the deafferented visual cortices are recruited for tactile information processing (e.g. [Sadato, 2002](http://www.ncbi.nlm.nih.gov/pubmed/12030824)), and hence, the brain actively adapts to the new source of information. So **no**, plastic changes are **not** the limiting factor.

Hence, the BrainPort (and other sensory substitution approaches like the SonicGuide etc.) is useful, portable, and not less importantly, the brain *actively adapts* in using the alternative visuo-tactile input by recruiting the visual cortex for processing visuotactile information. **Why then haven't sensory substitution approaches conquered the world of visual rehabilitation of the blind**"? There are a number of important downsides to sensory substitution devices (SSDs):

* SSDs **substitute** vision (by tactile or acoustic information), they do not **restore** it;
* The **limited resolution** of SSDs does not allow for detailed target analysis, i.e., discerning the difference between a fork and a knife will be very difficult;
* Instead, **relying on well-established coping mechanisms** and sticking to their usual routines may be far more effective in everyday life. For example, simply touching the fork or spoon will immediately tell them the difference, while scanning them with a SSD takes them ages (i.e., SSDs are not very practical);
* The restricted resolution of state-of-the-art SSDs also aids little in orientation and mobility (O&M). In other words, it does not help blind folks to go about. Instead, **established methods such as the guide cane and guide dog are far more effective for O&M**;
* Learning to use a SSD effectively takes *a lot of time*. **Not many people are willing to undergo an arduous rehabilitation program**;
* In case of the BrainPort - it is very visible to the outside world and **not very discreet** when a blind person is using it. It is known that 29% of consumers with disabilities abandon the use of assistive devices for reasons unrelated to device performance ([Phillips and Zhao, 1993](https://www.google.com.au/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CCIQFjAA&url=http%3A%2F%2Fcs.brynmawr.edu%2FCourses%2Fcs380%2Ffall2010%2Freadings%2FPhillips1993.pdf&ei=a73BVNysIM7n8AWqmoKACQ&usg=AFQjCNEeto4iQFmsrJQl_Gsz6XouCUuAAA&sig2=5qx3mDY7gEuhr20TvpnW0w&bvm=bv.84349003,d.dGc)).

**References**  
**-** [Bach-y-Rita 1969. *Nature* 221:963-4](http://www.nature.com/nature/journal/v221/n5184/abs/221963a0.html)  
**-** [Bach-y-Rita 2003. *JHCI* **15**:285-95](https://i.sstatic.net/fnJ33.jpg)  
**-** [Philips & Zhao 1993. *Assistive technology* **5**:36-45](https://www.google.com.au/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CCIQFjAA&url=http%3A%2F%2Fcs.brynmawr.edu%2FCourses%2Fcs380%2Ffall2010%2Freadings%2FPhillips1993.pdf&ei=a73BVNysIM7n8AWqmoKACQ&usg=AFQjCNEeto4iQFmsrJQl_Gsz6XouCUuAAA&sig2=5qx3mDY7gEuhr20TvpnW0w&bvm=bv.84349003,d.dGc)  
**-** [Sadato 2002. *Neuroimage* **16**:389-400](http://www.ncbi.nlm.nih.gov/pubmed/12030824)

> **Comments:**
> **username_0**: wow thanks Chris for this extensive overview of the current state of such devices!  

---

### Answer by username_1 on 2015-01-23. Score: 3

Chris gave an excellent answer for visual-to-tactile sensory substitution devices. I will not argue about the extent to which sensory substitution is successful, but the most widespread sensory substitution system for the blind is nowadays [The vOICe](http://www.username_1.com), a class of visual-to-auditory sensory substitution devices that is available globally. Its Android version has been installed over a quarter million times in the last few years, and there are a number of recent peer-reviewed [scientific publications](http://www.username_1.com/literature.htm) based on use of its Microsoft Windows version, covering various aspects of brain plasticity. Upcoming augmented reality glasses are likely to improve convenience of use. But yes, like Chris stated, learning to use a SSD effectively takes a lot of time.

> **Comments:**
> **username_2**: Nice answer! +1 are you related to The vOICe? I focused my question pretty much on Bach-y-Rita's legacy given the question. I think the main downsides of BrainPort, except for discreteness, also apply to other SSDs.  
> **username_1**: Thanks @Chris. Yes I'm related to The vOICe so I could be biased. However, The vOICe is far cheaper than the BrainPort device, offers a much higher resolution (e.g. 176 x 64 versus only 12 x 12 for the 144-electrode BrainPort), and it works with mass-market hardware. Oh, and you can speak and eat while using it. :-) On the other hand, the BrainPort should work better for people who are deaf-blind, and it may work better for tracking motion.  
> **username_2**: Will you be going to arvo :)  
> **username_1**: @Chris You asked the same question by e-mail last year. :-) Not this year, but there was an invited presentation on The vOICe at [Asia-ARVO 2013](http://www.username_1.com/extra/AsiaARVO2013_MeijerPBL.pdf) in New Delhi, India (PDF of slides).  
> **username_2**: thanks for the slides!! very helpful. Could you please provide a reference to the resolution you mention? I am writing this article and I can't find a decent source. Thanks.  
> **username_1**: The 176 x 64 hardware/software resolution is mentioned on the product pages for the [Microsoft Windows](http://www.username_1.com/winvoice.htm) and [Android](http://www.username_1.com/android.htm) versions of The vOICe. Perceptual resolution with The vOICe in the form of "visual" acuity is discussed in the peer-reviewed [PLOS ONE](http://dx.doi.org/10.1371/journal.pone.0033136) and [Frontiers in Psychology](http://dx.doi.org/10.3389/fpsyg.2013.00330) papers.  

---

### Answer by username_3 on 2015-01-23. Score: 0

[Tadoma](http://en.m.wikipedia.org/wiki/Tadoma) is a method of communicating used by some deaf blind individuals. The system was developed prior to 1967 and predates TVSS, although there is no hardware involved. I think this [video](https://m.youtube.com/watch?v=U_QMS-hzRMs) demonstrates the extent to which the approach is successful, and hence not limited by brain plasticity. Tadoma has fallen out of favor because it is difficult to learn and teach. There has been some research into vibrotactile hearing aids and there are some [commercially available](http://deafness.about.com/cs/vibrotactile/f/vibrotactile.htm).

> **Comments:**
> **username_2**: Tadoma is a non-electronic way of sensory substitution. Guide dogs and the guide cane pre-date the TVSS as well, obviously. The link http://deafness.about.com/cs/vibrotactile/f/vibrotactile.htm is quite obscure, as it links to dead websites, and weird unintelligible Chinese matter, unfortunately.  

---

