<p>No. You're asking about model selection. Larger sample sizes will allow you to choose more complex models. Read up on the keywords overfit/underfit, model selection, Structural Risk Minimization, and in particular this article:
<a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=705570" rel="nofollow noreferrer">https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=705570</a></p>

CC BY-SA 4.0
2019-02-26T11:33:26.643Z
My question is for fixed parameters set, do not consider about more complex models. Anyway, I will read this paper. Thank you very much
CSH
CC BY-SA 4.0
2019-02-26T12:09:46.317Z
If you use "number of layers" as a hyperparameter (as in your OP), this is NOT a fixed-parameter model, since more layers => more parameters.
Aryeh
CC BY-SA 4.0
2019-02-26T12:17:50.627Z
Sorry I didn't mention in details. Since we have hyperparameter sets $H_1$ to $H_n$, numbers of layers, for example, can be chosen only from those $n$ choices, but I found it is not a common way because as you said, we can construct more complex models to enhance the results.
CSH
CC BY-SA 4.0
2019-02-26T09:44:14.023Z
2019-02-26T09:44:14.023Z
Aryeh
<p>I'm new in this community and I don't know whether my question is proper for this community. I will delete this post if it is not proper.</p>

<p>I'm interested in deep learning network models and have a question about this.</p>

<p>Suppose we have a feedforward neural network models with different hyperparameters (for example, the number of layers, neurons and so on), say <span class="math-container">$H_1,..., H_n$</span>. Also assume all the samples are selected from a particular distribution.</p>

<p>First we got <span class="math-container">$100$</span> samples. If we find that a typical hyperparameter set, <span class="math-container">$H_i$</span>, performs best. Now I want to increase the number of samples to <span class="math-container">$1000, 10000,$</span> even millions. Is <span class="math-container">$H_i$</span> still the best hyperparameter set to perform among <span class="math-container">$H_i$</span>'s? I want to know whether there are references about such topics. The best is a mathematical proof about this fact.</p>

<p>Again, please tell me if this question is not proper to the community. Thanks a lot!</p>

<machine-learning><network-modeling><computational-mathematics>
Does fixed hyperparameters perform well regardless the number of training examples?
CSH
# Does fixed hyperparameters perform well regardless the number of training examples?

(Asked by: username_0 on 2019-02-26)

I'm new in this community and I don't know whether my question is proper for this community. I will delete this post if it is not proper.

I'm interested in deep learning network models and have a question about this.

Suppose we have a feedforward neural network models with different hyperparameters (for example, the number of layers, neurons and so on), say $H\_1,..., H\_n$. Also assume all the samples are selected from a particular distribution.

First we got $100$ samples. If we find that a typical hyperparameter set, $H\_i$, performs best. Now I want to increase the number of samples to $1000, 10000,$ even millions. Is $H\_i$ still the best hyperparameter set to perform among $H\_i$'s? I want to know whether there are references about such topics. The best is a mathematical proof about this fact.

Again, please tell me if this question is not proper to the community. Thanks a lot!

**Tags:** machine-learning, network-modeling, computational-mathematics


---

## 1 answers

### Answer by username_1 on 2019-02-26. Score: 1 | Accepted answer

No. You're asking about model selection. Larger sample sizes will allow you to choose more complex models. Read up on the keywords overfit/underfit, model selection, Structural Risk Minimization, and in particular this article:
<https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=705570>

> **Comments:**
> **username_0**: My question is for fixed parameters set, do not consider about more complex models. Anyway, I will read this paper. Thank you very much  
> **username_1**: If you use "number of layers" as a hyperparameter (as in your OP), this is NOT a fixed-parameter model, since more layers => more parameters.  
> **username_0**: Sorry I didn't mention in details. Since we have hyperparameter sets $H_1$ to $H_n$, numbers of layers, for example, can be chosen only from those $n$ choices, but I found it is not a common way because as you said, we can construct more complex models to enhance the results.  

---

