<p>I assume we're discussing simple feed-forward neural nets, i.e. multi-layer perceptrons.</p>

<ol>
<li>The hidden layer is necessary to capture non-linear dependencies between your data's features and the variable you're trying to predict. If you don't use a hidden layer, you might as well use linear regression (for regression) or <a href="https://en.wikipedia.org/wiki/Logistic_regression" rel="nofollow">logistic regression</a> (for classification).</li>
<li>By trying various numbers of hidden layers and evaluating how well they work, e.g. in a cross-validation setting. Commonly, one hidden layer will be enough and NN performance is optimized by varying its size and the regularization.</li>
</ol>

<p>Note that with more than two hidden layers, you're in deep learning land and you probably need custom algorithms to train your net. The reason is that vanilla backpropagation suffers from the "vanishing gradient" problem in deep nets: the gradient of the error function dies down at the layers close to the input, and those layers will hardly be trained.</p>

CC BY-SA 3.0
2012-01-03T16:59:06.990Z
2013-12-26T12:28:45.630Z
2013-12-26T12:28:45.630Z
Fred Foo
<p>A hidden layer is used to increase the expressiveness of the network.  It allows the network to represent more complex models than possible without the hidden layer.</p>

<p>Choosing the number of hidden layers, or more generally choosing your network architecture including the number of hidden units in hidden layers as well, are decisions that should be based on your training and cross-validation data.  You should train the network with a set amount of nodes (to start, try one hidden layer, with one unit per input unit) and test the model.</p>

<p>See this link for more help:  <a href="http://www.faqs.org/faqs/ai-faq/neural-nets/part3/section-10.html">http://www.faqs.org/faqs/ai-faq/neural-nets/part3/section-10.html</a></p>

CC BY-SA 3.0
2012-01-03T21:01:04.990Z
that link is quite helpful
erik
CC BY-SA 3.0
2012-01-03T17:02:11.917Z
2012-01-03T17:02:11.917Z
Neal Tibrewala
<p>Following up on this <a href="https://cstheory.stackexchange.com/questions/1515/what-does-number-of-inputs-to-each-neuron-mean-in-neural-network-terms">question</a>...</p>

<p>I am attempting to learn how to use and create neural networks for my research, and one point is somewhat escaping me.  I realize that hidden layers are a somewhat necessary portion of this, however I am stuck on two points which my references aren't explaining to my satisfaction:</p>

<ol>
<li>What is the exact purpose of the hidden layer?</li>
<li>How does one determine how many hidden layers to use?</li>
</ol>

<p>From what I gather, it is to "model" real world functionality, but if possible I'd like a bit more of an explanation.</p>

<ne.neural-evol>
Purpose and determining when to use hidden layers
erik
# Purpose and determining when to use hidden layers

(Asked by: username_0 on 2012-01-03)

Following up on this [question](https://cstheory.stackexchange.com/questions/1515/what-does-number-of-inputs-to-each-neuron-mean-in-neural-network-terms)...

I am attempting to learn how to use and create neural networks for my research, and one point is somewhat escaping me. I realize that hidden layers are a somewhat necessary portion of this, however I am stuck on two points which my references aren't explaining to my satisfaction:

1. What is the exact purpose of the hidden layer?
2. How does one determine how many hidden layers to use?

From what I gather, it is to "model" real world functionality, but if possible I'd like a bit more of an explanation.

**Tags:** ne.neural-evol


---

## 2 answers

### Answer by username_2 on 2012-01-03. Score: 9 | Accepted answer

A hidden layer is used to increase the expressiveness of the network. It allows the network to represent more complex models than possible without the hidden layer.

Choosing the number of hidden layers, or more generally choosing your network architecture including the number of hidden units in hidden layers as well, are decisions that should be based on your training and cross-validation data. You should train the network with a set amount of nodes (to start, try one hidden layer, with one unit per input unit) and test the model.

See this link for more help: <http://www.faqs.org/faqs/ai-faq/neural-nets/part3/section-10.html>

> **Comments:**
> **username_0**: that link is quite helpful  

---

### Answer by username_1 on 2012-01-03. Score: 6

I assume we're discussing simple feed-forward neural nets, i.e. multi-layer perceptrons.

1. The hidden layer is necessary to capture non-linear dependencies between your data's features and the variable you're trying to predict. If you don't use a hidden layer, you might as well use linear regression (for regression) or [logistic regression](https://en.wikipedia.org/wiki/Logistic_regression) (for classification).
2. By trying various numbers of hidden layers and evaluating how well they work, e.g. in a cross-validation setting. Commonly, one hidden layer will be enough and NN performance is optimized by varying its size and the regularization.

Note that with more than two hidden layers, you're in deep learning land and you probably need custom algorithms to train your net. The reason is that vanilla backpropagation suffers from the "vanishing gradient" problem in deep nets: the gradient of the error function dies down at the layers close to the input, and those layers will hardly be trained.


---

