<p>To my knowledge, there is no adjusted RMSD.</p>

<p>RMSD, unlike $R^2$, isn't typically used to compare models across the literature. $R^2$ represents the proportion of variance explained by the model, a construct which translates well across different experimental designs. Adjusted $R^2$ distorts this by accounting for the number of parameters in your model, but is a better estimate of the proportion of variance in the <em>population</em> explained by your model.</p>

<p>RMSD is in whatever units you happen to be using. It should be interpreted against some benchmark for what you think a good model should be, or whatever has been deemed an "acceptable error rate". It's not clear to me that adjusting for the number of parameters in RMSD would provide a number that is inherently meaningful, but I'm not sure if this is true.</p>

<p>If you would like to select among several model candidates, you may want to compute <a href="http://en.wikipedia.org/wiki/Akaike_information_criterion">AIC</a> (Akaike Information Criterion) or <a href="http://en.wikipedia.org/wiki/Bayesian_information_criterion">BIC</a> (Bayesian Information Criterion), both of which are attempts to maximize the likelihood of your model while minimizing the number of parameters. Both metrics are commonly used in cognitive modeling, and I think this may be what you are looking for.</p>

<p>While I think this question is on-topic here, you will likely get better answers on <a href="http://stats.stackexchange.com">stats.stackexchange.com</a> (as Jeromy points out) because your question is not really specific to <em>cognitive</em> modeling.</p>

CC BY-SA 3.0
2013-02-08T23:40:04.340Z
2013-02-08T23:40:04.340Z
Jeff
<p>There are at least 3 ways to discount SSE (or RMSE) by the number of free params:</p>

<p>$$
\text{adjusted RMSE} = \sqrt{\frac{SSE}{n - k}}
$$</p>

<p>$$
AIC = n \times ln\left(\frac{SSE}{n}\right) - k \times ln(n)
$$</p>

<p>$$
BIC = n \times ln\left(\frac{SSE}{n}\right) - 2 \times k
$$</p>

<p>or in computer code style:</p>

<pre><code>k = number of free params
n = number of DV's
SSE = sum of squared errors

adjusted RMSE = sqrt ( SSE / n - k )
AIC = n * ln(SSE/n) - k * ln(n)
BIC = n * ln(SSE/n) - 2 * k
</code></pre>

CC BY-SA 3.0
2013-02-14T14:22:55.617Z
2013-02-17T08:45:34.357Z
2013-02-17T08:45:34.357Z
vdv
<p>How do I adjust SSE (sum of squared errors) or RMSE (<a href="http://en.wikipedia.org/wiki/Root-mean-square_deviation" rel="nofollow">root-mean-square errors</a>) for the number of free parameters in the model?</p>

<p>Is there an "adjusted" RMSD metric similar to the <a href="http://en.wikipedia.org/wiki/Coefficient_of_determination#Adjusted_R2" rel="nofollow">adjusted r-squared metric</a>?</p>

CC BY-SA 3.0
2013-02-08T05:04:01.627Z
please don't assume that we know what RMSD stands for. The more effort you invest in carefully phrasing your question, the more motivated people will feel to answer it.
Artem Kaznatcheev
CC BY-SA 3.0
2013-02-08T05:13:48.850Z
This sounds more like a question better suited for http://stats.stackexchange.com/ . Is there an aspect of this that relates particularly to cognitive modelling?
Jeromy Anglim
CC BY-SA 3.0
2013-02-08T14:30:10.863Z
Not sure to understand what you mean by "adjusted" here ? Do you mean adjusted as for "goodness of fit" or as correction in regards to some parameters ?
Cheatboy2
CC BY-SA 3.0
2013-02-08T23:17:50.717Z
@Cheatboy2 I believe he is referring to a correction in the number of parameters, as in adjusted $r^2$
Jeff
<cognitive-modeling><computational-modeling><statistics>
How to adjust SSE or RMSE for the number of free parameters in the model?
vdv
# How to adjust SSE or RMSE for the number of free parameters in the model?

(Asked by: username_0 on 2013-02-08)

How do I adjust SSE (sum of squared errors) or RMSE ([root-mean-square errors](http://en.wikipedia.org/wiki/Root-mean-square_deviation)) for the number of free parameters in the model?

Is there an "adjusted" RMSD metric similar to the [adjusted r-squared metric](http://en.wikipedia.org/wiki/Coefficient_of_determination#Adjusted_R2)?

**Tags:** cognitive-modeling, computational-modeling, statistics

> **Comments:**
> **username_1**: please don't assume that we know what RMSD stands for. The more effort you invest in carefully phrasing your question, the more motivated people will feel to answer it.  
> **username_2**: This sounds more like a question better suited for http://stats.stackexchange.com/ . Is there an aspect of this that relates particularly to cognitive modelling?  
> **username_3**: Not sure to understand what you mean by "adjusted" here ? Do you mean adjusted as for "goodness of fit" or as correction in regards to some parameters ?  
> **username_4**: @username_3 I believe he is referring to a correction in the number of parameters, as in adjusted $r^2$  

---

## 2 answers

### Answer by username_0 on 2013-02-14. Score: 3 | Accepted answer

There are at least 3 ways to discount SSE (or RMSE) by the number of free params:
$$
\text{adjusted RMSE} = \sqrt{\frac{SSE}{n - k}}
$$
$$
AIC = n \times ln\left(\frac{SSE}{n}\right) - k \times ln(n)
$$
$$
BIC = n \times ln\left(\frac{SSE}{n}\right) - 2 \times k
$$
or in computer code style:
k = number of free params
n = number of DV's
SSE = sum of squared errors

adjusted RMSE = sqrt ( SSE / n - k )
AIC = n * ln(SSE/n) - k * ln(n)
BIC = n * ln(SSE/n) - 2 * k


---

### Answer by username_4 on 2013-02-08. Score: 5

To my knowledge, there is no adjusted RMSD.

RMSD, unlike $R^2$, isn't typically used to compare models across the literature. $R^2$ represents the proportion of variance explained by the model, a construct which translates well across different experimental designs. Adjusted $R^2$ distorts this by accounting for the number of parameters in your model, but is a better estimate of the proportion of variance in the *population* explained by your model.

RMSD is in whatever units you happen to be using. It should be interpreted against some benchmark for what you think a good model should be, or whatever has been deemed an "acceptable error rate". It's not clear to me that adjusting for the number of parameters in RMSD would provide a number that is inherently meaningful, but I'm not sure if this is true.

If you would like to select among several model candidates, you may want to compute [AIC](http://en.wikipedia.org/wiki/Akaike_information_criterion) (Akaike Information Criterion) or [BIC](http://en.wikipedia.org/wiki/Bayesian_information_criterion) (Bayesian Information Criterion), both of which are attempts to maximize the likelihood of your model while minimizing the number of parameters. Both metrics are commonly used in cognitive modeling, and I think this may be what you are looking for.

While I think this question is on-topic here, you will likely get better answers on [stats.stackexchange.com](http://stats.stackexchange.com) (as Jeromy points out) because your question is not really specific to *cognitive* modeling.


---

