<p>Venturing a contrasting view: VC dimension as cited by AK is a good/solid theoretical measurement of ANN complexity but would be unlikely to be applied to any real constructed ANN except as an estimate, and researchers/papers in large applied ANNs do not currently estimate VC dimension.</p>

<p>In a sense, how to measure complexity of an ANN is an open question that researchers are currently attempting to answer ("work in progress") and won't be solved until there is some more general theory of <a href="http://en.wikipedia.org/wiki/Feature_detection" rel="nofollow">"feature detection"</a> which seems to be slowly emerging at the moment e.g., in <a href="http://en.wikipedia.org/wiki/Deep_learning" rel="nofollow">deep learning</a> research. such a theory is likely a long time in the making if it is even possible &amp; ever is obtained. Roughly, in this view, a "more complex" ANN recognizes "more complex" features across different dimensions (spatial, temporal, different sensory modalities such as auditory, kinesthetic (robotics), etc).</p>

<p>It is worthwhile &amp; fairly objective however in the meantime to just consider a "black box" or "operational" style estimate based on intelligent functionality exhibited by the ANN. in other words, what can the ANN accomplish, and how does this compare to our only other benchmark of intelligence, namely biological?</p>

<p>You tend to rule this out in the question, but there is already a commonly used informal "sliding scale" of biological intelligence, e.g., with say small organisms at one end, moving through insects and mammals, etc., and humans at the other end. In animal science there are fairly conclusive questions &amp; study to, e.g., "which is smarter, a dog, a pig, or a cat" with fairly nuanced/definitive answers (also with understanding that "context matters" &amp; there are some various aspects of incomparability).</p>

<p>This approach basically dates to the <a href="http://en.wikipedia.org/wiki/Turing_test" rel="nofollow">Turing test</a> and the Turing test is still a very valid scientific measurement of intelligence, still applied, e.g., in the <a href="http://en.wikipedia.org/wiki/Loebner_contest" rel="nofollow">Loebner contest</a>, &amp; seems to have roots also in <a href="http://en.wikipedia.org/wiki/Behaviorism" rel="nofollow">behavioralist psychology</a> principles. it involves the basic aspects of a scientific test such as a <a href="http://en.wikipedia.org/wiki/Scientific_control" rel="nofollow">control</a> &amp; <a href="http://en.wikipedia.org/wiki/Blind_experiment" rel="nofollow">blind sampling</a> etcetera.</p>

<p>Moreover, there are aspects of intelligence that are unique to humans such as speech recognition, image recognition, language translation, etc., and these lead to good benchmarks of ANNs that are targeted at human-like functionality as far as how well the ANN compares to human performance. it can even lead to measurements of supra-human performance in various cases.</p>

<p>This does not lead to a single quantifiable/numerical estimate of intelligence but in psychology, that premise is starting to be quite seriously questioned anyway, e.g., <a href="http://en.wikipedia.org/wiki/Theory_of_multiple_intelligences" rel="nofollow">theory of multiple intelligences</a> &amp; perhaps even refuted somewhat at this point.</p>

CC BY-SA 3.0
2014-02-09T02:36:55.140Z
note that in biology intelligence basically scales with # of neurons, and so # of neurons is probably a rough/good estimate of complexity _if they are functioning at least as effectively as in the biological case_ (a _caveat_ which basically rules out all ANNs in current existence—also strongly disagreeing with the hyperbolic and unjustified claim you cite that a large ANN has successfully replicated cat-level intelligence).
vzn
CC BY-SA 3.0
2014-02-09T02:44:41.107Z
In terms of number of neurons, our gut has nearly as many as the human brain.  If anything, synapses would be a better indicator of intelligence, but that's not always true.
Chuck Sherrington
CC BY-SA 3.0
2014-02-09T03:24:08.933Z
?!? there are roughly ~$10^{10}$ neurons in the human brain. hadnt heard about much in the stomach... do you have a ref for that? sounds incorrect to me
vzn
CC BY-SA 3.0
2014-02-09T03:26:47.337Z
http://en.wikipedia.org/wiki/Enteric_nervous_system#Anatomy there are actually 10^8 neurons, but not much "intelligence"
Chuck Sherrington
CC BY-SA 3.0
2014-02-09T03:39:10.897Z
ok good (counter)pt but seems that many neurons must be doing something significant, maybe optimizing digestion breakdown for some process(es) that require significant complexity... not intelligent by typical stds but that fits into the answer pt about "different kinds of intelligences"... thx for that!
vzn
CC BY-SA 3.0
2016-09-14T21:24:34.450Z
The Turing Test may still be applied, but it cannot possibly be a "very valid scientific measurement of intelligence." If one mistakes a mannequin for a human being, does that make the mannequin human? No. Nor does mistaking a computer for a human make the computer intelligent. It's a spectacular violation of common sense, one that measures the gullibility of observers more than anything else. If anyone but Turing said it, it would be laughed off. Turing was a brilliant man in other respects, but this test was hardly one of his better moments.
SQLServerSteve
CC BY-SA 3.0
2017-11-07T14:24:50.027Z
For posterity's sake, the wikipedia article now cites that the "gut" has ~$500$ million neurons, nowhere near ~$10^{10}$.
vkehayas
CC BY-SA 3.0
2014-02-09T02:27:20.010Z
2014-02-09T02:43:14.820Z
2014-02-09T02:43:14.820Z
vzn
<p>The standard complexity metric in theoretical computer science and machine learning, in particular in statistical learning theory, is the <a href="http://en.wikipedia.org/wiki/VC_dimension">Vapnik–Chervonenkis (VC) dimension</a>. It is of interest because it gives us a very good tool to measure the learning ability of a neural network (or any other statistical learner, in general).</p>

<p>A good introduction to the use of VC dimension for studying neural nets is:</p>

<p>Eduardo D. Sontag [1998] "VC dimension of neural networks" [<a href="http://mathsci.kaist.ac.kr/~nipl/mas557/VCD_ANN_3.pdf">pdf</a>].</p>

<p>There, the author shows (for instance) that a network with one hidden layer, $n$ inputs, and $\tanh$ neurons has VC dimension of $n + 1$. He also explain some basic technique for how to upper-bound the VC dimension, and for how to use it for dynamic neural nets.</p>

CC BY-SA 3.0
2012-02-08T23:41:05.113Z
I asked a [follow up question](http://stats.stackexchange.com/q/23484/6516) on ML.SE about alternatives to VC-dimension used in machine learning.
Artem Kaznatcheev
CC BY-SA 3.0
2012-02-03T03:15:27.830Z
2012-06-29T03:25:17.360Z
2012-06-29T03:25:17.360Z
Artem Kaznatcheev
<p>After asking the question <a href="https://cogsci.stackexchange.com/questions/245/what-is-the-most-complex-artificial-neural-network-created-to-date">What is the most complex neural network...</a> I realized I don't really have a good metric of "complexity" in a general sense. The simplest measure would likely be count of neurons or number of synapses, but that fails to take into account the structure of the network.</p>

<p>A couple measures of complexity are discussed in the paper <a href="http://math.bu.edu/people/mkon/pppp11.pdf" rel="nofollow noreferrer">Complexity of Predictive Neural Networks
</a> but they are very specific to a single task. One is the amount of work needed to learn a certain thing, and the other is how many neurons are needed to approximate a certain function.</p>

<p>Rough, animal based measures are often employees for the sake of grabbing headlines; such as the incorrect claims that <a href="http://bluebrain.epfl.ch/" rel="nofollow noreferrer">The Blue Brain Project</a> had emulated a neural network "as complex as" a cat's brain. C. elegans is a <a href="http://www.csi.uoregon.edu/projects/celegans/" rel="nofollow noreferrer">common and seemingly attainable</a> level of complexity for an artificial neural network. </p>

<p>Animal based measures are relateable to the layman but seem questionable, especially when comparing a neural network to that of an animal who's neural network has not been completely mapped (<a href="http://ims.dse.ibaraki.ac.jp/research/C_elegans_en.html" rel="nofollow noreferrer">as C. elegans has</a>).</p>

<p>What is a meaningful measure by which artificial neural networks can be measured? How are such networks currently compared? Can any such metric appropriately measure complexity of such a system?</p>

CC BY-SA 3.0
2012-02-08T23:51:04.633Z
For a course grained and asymptotic computational-complexity metric see [this cstheory question](http://cstheory.stackexchange.com/q/2698/1037)
Artem Kaznatcheev
<terminology><measurement><neural-network><theoretical-neuroscience>
What is an effective metric of complexity for an Artificial Neural Network?
Zelda
# What is an effective metric of complexity for an Artificial Neural Network?

(Asked by: username_0 on 2012-02-03)

After asking the question [What is the most complex neural network...](https://cogsci.stackexchange.com/questions/245/what-is-the-most-complex-artificial-neural-network-created-to-date) I realized I don't really have a good metric of "complexity" in a general sense. The simplest measure would likely be count of neurons or number of synapses, but that fails to take into account the structure of the network.

A couple measures of complexity are discussed in the paper [Complexity of Predictive Neural Networks](http://math.bu.edu/people/mkon/pppp11.pdf) but they are very specific to a single task. One is the amount of work needed to learn a certain thing, and the other is how many neurons are needed to approximate a certain function.

Rough, animal based measures are often employees for the sake of grabbing headlines; such as the incorrect claims that [The Blue Brain Project](http://bluebrain.epfl.ch/) had emulated a neural network "as complex as" a cat's brain. C. elegans is a [common and seemingly attainable](http://www.csi.uoregon.edu/projects/celegans/) level of complexity for an artificial neural network.

Animal based measures are relateable to the layman but seem questionable, especially when comparing a neural network to that of an animal who's neural network has not been completely mapped ([as C. elegans has](http://ims.dse.ibaraki.ac.jp/research/C_elegans_en.html)).

What is a meaningful measure by which artificial neural networks can be measured? How are such networks currently compared? Can any such metric appropriately measure complexity of such a system?

**Tags:** terminology, measurement, neural-network, theoretical-neuroscience

> **Comments:**
> **username_1**: For a course grained and asymptotic computational-complexity metric see [this cstheory question](http://cstheory.stackexchange.com/q/2698/1037)  

---

## 2 answers

### Answer by username_1 on 2012-02-03. Score: 15 | Accepted answer

The standard complexity metric in theoretical computer science and machine learning, in particular in statistical learning theory, is the [Vapnik–Chervonenkis (VC) dimension](http://en.wikipedia.org/wiki/VC_dimension). It is of interest because it gives us a very good tool to measure the learning ability of a neural network (or any other statistical learner, in general).

A good introduction to the use of VC dimension for studying neural nets is:

Eduardo D. Sontag [1998] "VC dimension of neural networks" [[pdf](http://mathsci.kaist.ac.kr/~nipl/mas557/VCD_ANN_3.pdf)].

There, the author shows (for instance) that a network with one hidden layer, $n$ inputs, and $\tanh$ neurons has VC dimension of $n + 1$. He also explain some basic technique for how to upper-bound the VC dimension, and for how to use it for dynamic neural nets.

> **Comments:**
> **username_1**: I asked a [follow up question](http://stats.stackexchange.com/q/23484/6516) on ML.SE about alternatives to VC-dimension used in machine learning.  

---

### Answer by username_2 on 2014-02-09. Score: 1

Venturing a contrasting view: VC dimension as cited by AK is a good/solid theoretical measurement of ANN complexity but would be unlikely to be applied to any real constructed ANN except as an estimate, and researchers/papers in large applied ANNs do not currently estimate VC dimension.

In a sense, how to measure complexity of an ANN is an open question that researchers are currently attempting to answer ("work in progress") and won't be solved until there is some more general theory of ["feature detection"](http://en.wikipedia.org/wiki/Feature_detection) which seems to be slowly emerging at the moment e.g., in [deep learning](http://en.wikipedia.org/wiki/Deep_learning) research. such a theory is likely a long time in the making if it is even possible & ever is obtained. Roughly, in this view, a "more complex" ANN recognizes "more complex" features across different dimensions (spatial, temporal, different sensory modalities such as auditory, kinesthetic (robotics), etc).

It is worthwhile & fairly objective however in the meantime to just consider a "black box" or "operational" style estimate based on intelligent functionality exhibited by the ANN. in other words, what can the ANN accomplish, and how does this compare to our only other benchmark of intelligence, namely biological?

You tend to rule this out in the question, but there is already a commonly used informal "sliding scale" of biological intelligence, e.g., with say small organisms at one end, moving through insects and mammals, etc., and humans at the other end. In animal science there are fairly conclusive questions & study to, e.g., "which is smarter, a dog, a pig, or a cat" with fairly nuanced/definitive answers (also with understanding that "context matters" & there are some various aspects of incomparability).

This approach basically dates to the [Turing test](http://en.wikipedia.org/wiki/Turing_test) and the Turing test is still a very valid scientific measurement of intelligence, still applied, e.g., in the [Loebner contest](http://en.wikipedia.org/wiki/Loebner_contest), & seems to have roots also in [behavioralist psychology](http://en.wikipedia.org/wiki/Behaviorism) principles. it involves the basic aspects of a scientific test such as a [control](http://en.wikipedia.org/wiki/Scientific_control) & [blind sampling](http://en.wikipedia.org/wiki/Blind_experiment) etcetera.

Moreover, there are aspects of intelligence that are unique to humans such as speech recognition, image recognition, language translation, etc., and these lead to good benchmarks of ANNs that are targeted at human-like functionality as far as how well the ANN compares to human performance. it can even lead to measurements of supra-human performance in various cases.

This does not lead to a single quantifiable/numerical estimate of intelligence but in psychology, that premise is starting to be quite seriously questioned anyway, e.g., [theory of multiple intelligences](http://en.wikipedia.org/wiki/Theory_of_multiple_intelligences) & perhaps even refuted somewhat at this point.

> **Comments:**
> **username_2**: note that in biology intelligence basically scales with # of neurons, and so # of neurons is probably a rough/good estimate of complexity _if they are functioning at least as effectively as in the biological case_ (a _caveat_ which basically rules out all ANNs in current existence—also strongly disagreeing with the hyperbolic and unjustified claim you cite that a large ANN has successfully replicated cat-level intelligence).  
> **username_3**: In terms of number of neurons, our gut has nearly as many as the human brain.  If anything, synapses would be a better indicator of intelligence, but that's not always true.  
> **username_2**: ?!? there are roughly ~$10^{10}$ neurons in the human brain. hadnt heard about much in the stomach... do you have a ref for that? sounds incorrect to me  
> **username_3**: http://en.wikipedia.org/wiki/Enteric_nervous_system#Anatomy there are actually 10^8 neurons, but not much "intelligence"  
> **username_2**: ok good (counter)pt but seems that many neurons must be doing something significant, maybe optimizing digestion breakdown for some process(es) that require significant complexity... not intelligent by typical stds but that fits into the answer pt about "different kinds of intelligences"... thx for that!  
> **username_4**: The Turing Test may still be applied, but it cannot possibly be a "very valid scientific measurement of intelligence." If one mistakes a mannequin for a human being, does that make the mannequin human? No. Nor does mistaking a computer for a human make the computer intelligent. It's a spectacular violation of common sense, one that measures the gullibility of observers more than anything else. If anyone but Turing said it, it would be laughed off. Turing was a brilliant man in other respects, but this test was hardly one of his better moments.  
> **username_5**: For posterity's sake, the wikipedia article now cites that the "gut" has ~$500$ million neurons, nowhere near ~$10^{10}$.  

---

