<p>From a theoretical computer science point of view, is there a lower limit on the number of hours of speech needed to train a neural net to translate speech to text? <a href="https://www.quora.com/How-many-voice-data-is-needed-for-the-training-of-commercial-DNN-HMM-ASR-system-with-accuracy-of-90" rel="nofollow noreferrer">An estimate from CMU</a> is 3000-5000 hours for 90% accuracy
commercial quality speech recognition. Is there is a minimum amount of information needed to reproduce the complexity of the actual language.  I.e., if you had 5000 hours and compress it down with a neural net, there is some minimum size neural net needed to do a good job at speech recognition.   You can call that the "bit complexity of a natural language".  Does inverting the compression ratio tell you how many hours of speech, at a minimum, you would need to train a commercial quality speech recognition system?  For context, the classic paper <a href="https://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf" rel="nofollow noreferrer">Prediction and Entropy of Printed English</a> seems relevant.  </p>

<p>Also relevant is the work of <a href="http://mltheory.cs.princeton.edu/" rel="nofollow noreferrer">Sanjeev Arora</a> on <a href="http://www.math.ias.edu/theoretical_machine_learning" rel="nofollow noreferrer">theoretical machine learning</a>.</p>

<p>This question comes out of the <a href="https://www.nist.gov/itl/iad/mig/openclir-evaluation" rel="nofollow noreferrer">NIST OpenCLIR19 evaluation</a>  and <a href="https://www.nist.gov/itl/iad/mig/opensat" rel="nofollow noreferrer">OpenSAT19</a>.  OpenSAT is having an <a href="https://www.nist.gov/news-events/events/2019/08/opensat19-workshop" rel="nofollow noreferrer">open meeting in August</a>.  In both programs there is a focus on <a href="https://www.quora.com/Who-is-doing-interesting-NLP-research-for-low-resource-languages" rel="nofollow noreferrer">Low Resource Languages</a>. OpenCLIR had 80 hours of Swahili and OpenSAT has Pashto. In OpenCLIR, no solver team was able to do speech recognition on Swahili training from 80 hours of speech. Hence the question.</p>

CC BY-SA 4.0
2019-07-08T23:00:47.857Z
No.  This is an empirical question, not one that can be answered by standard CS theory.
D.W.
CC BY-SA 4.0
2019-07-08T23:06:13.110Z
You can formalize the content of a language as a set of grammatical rules for the output and a set of instructions for how to make a vocal tract machine generate the input, and then it's not quite as empirical.  That is, you can transfer the essence of the problem from a physical setting to an abstract one.
Lars Ericson
CC BY-SA 4.0
2019-07-09T00:51:56.600Z
I disagree that this is an empirical problem. The OP clearly asks a problem that can be formalized in the language of information theoretic and complexity theoretic machine learning.
Turbo
CC BY-SA 4.0
2019-07-09T17:51:22.320Z
I agree that this is mostly an empirical question in its current form. It is not clear how  speech and grammar are supposed to be formalized. The question asks for hours of training, which is clearly machine dependant. There is some question about minimum network size, but again it's not clear how this is formalized. OP argues that it *can* be formalized, and I don't deny that, but I think StackExchange is not meant for such an open ended question. The expectation is that the question is already formulated in mathematical terms.
Sasho Nikolov
CC BY-SA 4.0
2019-07-09T20:38:51.933Z
@SashoNikolov complexity theory is working it's way into the machine learning space, for example http://proceedings.mlr.press/v32/arora14.pdf.
Lars Ericson
CC BY-SA 4.0
2019-07-09T23:52:38.953Z
I agree. But that doesn't change my answer.
Sasho Nikolov
<machine-learning><it.information-theory><communication-complexity><shannon-entropy><shannon>
Minimum number of hours of speech needed to train a neural net to recognize speech
Lars Ericson
# Minimum number of hours of speech needed to train a neural net to recognize speech

(Asked by: username_0 on 2019-07-08)

From a theoretical computer science point of view, is there a lower limit on the number of hours of speech needed to train a neural net to translate speech to text? [An estimate from CMU](https://www.quora.com/How-many-voice-data-is-needed-for-the-training-of-commercial-DNN-HMM-ASR-system-with-accuracy-of-90) is 3000-5000 hours for 90% accuracy
commercial quality speech recognition. Is there is a minimum amount of information needed to reproduce the complexity of the actual language. I.e., if you had 5000 hours and compress it down with a neural net, there is some minimum size neural net needed to do a good job at speech recognition. You can call that the "bit complexity of a natural language". Does inverting the compression ratio tell you how many hours of speech, at a minimum, you would need to train a commercial quality speech recognition system? For context, the classic paper [Prediction and Entropy of Printed English](https://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf) seems relevant.

Also relevant is the work of [Sanjeev Arora](http://mltheory.cs.princeton.edu/) on [theoretical machine learning](http://www.math.ias.edu/theoretical_machine_learning).

This question comes out of the [NIST OpenCLIR19 evaluation](https://www.nist.gov/itl/iad/mig/openclir-evaluation) and [OpenSAT19](https://www.nist.gov/itl/iad/mig/opensat). OpenSAT is having an [open meeting in August](https://www.nist.gov/news-events/events/2019/08/opensat19-workshop). In both programs there is a focus on [Low Resource Languages](https://www.quora.com/Who-is-doing-interesting-NLP-research-for-low-resource-languages). OpenCLIR had 80 hours of Swahili and OpenSAT has Pashto. In OpenCLIR, no solver team was able to do speech recognition on Swahili training from 80 hours of speech. Hence the question.

**Tags:** machine-learning, it.information-theory, communication-complexity, shannon-entropy, shannon

> **Comments:**
> **username_1**: No.  This is an empirical question, not one that can be answered by standard CS theory.  
> **username_0**: You can formalize the content of a language as a set of grammatical rules for the output and a set of instructions for how to make a vocal tract machine generate the input, and then it's not quite as empirical.  That is, you can transfer the essence of the problem from a physical setting to an abstract one.  
> **username_2**: I disagree that this is an empirical problem. The OP clearly asks a problem that can be formalized in the language of information theoretic and complexity theoretic machine learning.  
> **username_3**: I agree that this is mostly an empirical question in its current form. It is not clear how  speech and grammar are supposed to be formalized. The question asks for hours of training, which is clearly machine dependant. There is some question about minimum network size, but again it's not clear how this is formalized. OP argues that it *can* be formalized, and I don't deny that, but I think StackExchange is not meant for such an open ended question. The expectation is that the question is already formulated in mathematical terms.  
> **username_0**: @SashoNikolov complexity theory is working it's way into the machine learning space, for example http://proceedings.mlr.press/v32/arora14.pdf.  
> **username_3**: I agree. But that doesn't change my answer.  

---

## 0 answers

