<p>The chi-squared should always decrease as you increase the complexity of the model (in this case by increasing the number of polynomial terms). The <em>reduced</em> chi-squared may decrease or increase, because the reduced chi-squared is the chi-squared divided by the number of degrees of freedom, but you have reduced the number of degrees of freedom.</p>
<p>Whether the additional complexity is warranted can be checked by comparing the change in the chi-squared against the number of degrees of freedom in the fit. For example you could use the <a href="https://en.wikipedia.org/wiki/Bayesian_information_criterion" rel="nofollow noreferrer">Bayesian Information Criterion (BIC)</a> but generally speaking, if the reduced chi-squared does not decrease then you are probably over-fitting the data.</p>

CC BY-SA 4.0
2023-01-13T11:59:38.870Z
Another way to put the latter part of the final sentence is that the point at which the reduced chi-squared does not decrease significantly from the previous iteration with one fewer degree of freedom is the point at which one might well be overfitting.
David Hammen
CC BY-SA 4.0
2023-01-13T12:42:17.740Z
@DavidHammen it can be more complicated. For example the true function could be odd or even in which case adding even or odd polynomial terms won't change the chi-squared and the reduced chi-squared will increase. But adding a further term could then see a significant decrease.
ProfRob
CC BY-SA 4.0
2023-01-13T12:55:50.963Z
That is correct. Sometimes it's better to take a stepwise approach of looking at each model parameter individually and adding the one that results in the best improvement to the model, repeating until there are no modeling parameters left or the improvement is negligible. An alternative is to start with using all modeling parameters on the first round and then eliminating modeling parameters one by one based on some measure of statistical (in)significance, stopping the trimming when removing any one of the remaining parameters would trim too much (from a statistically significant perspective).
David Hammen
CC BY-SA 4.0
2023-01-12T20:24:47.807Z
2023-01-12T20:24:47.807Z
ProfRob
<blockquote>
<p>Should I expect a decrease in the reduced chi-square values with increasing n?</p>
</blockquote>
<p>You should see a decrease up to the point where you start overfitting -- assuming that your assumption that the correct model is some kind of polynomial is correct. If it's not a polynomial, adding ever more terms might by luck capture ever more of the unmodeled variation.</p>
<p>Suppose one has captured data at <span class="math-container">$n&gt;4$</span> points along some domain of interest, each with a bit of measurement error. Suppose the true model is a quadratic. The regression will improve with each step in going from a constant to a line to a quadratic to a cubic, so the chi-squared value will decrease with each step (but not much for the last step from a quadratic to a cubic), The reduced chi-squared value will decrease nicely on moving from a constant to a line to a quadratic. It won't reduce by much, or it might even increase on that last step from a quadratic to a cubic. That's a sign of overfitting (and also of using the wrong model).</p>
<p>Suppose one has lots of measurements (e.g., <span class="math-container">$n=1000$</span>) and the measurement noise is not particularly small. Going from a constant to a line to a quadratic still results in a nice stepwise decrease in the reduced chi-square value. Here the step to a cubic might also result in a decrease because the addition of the cubic term might happen to capture some of the measurement noise and because dividing by 996 is not all that different from dividing by 997.</p>

CC BY-SA 4.0
2023-01-13T12:21:20.410Z
2023-01-13T12:29:26.670Z
2023-01-13T12:29:26.670Z
David Hammen
<p>I am currently plotting a continuum of observed data. I need to check the effectiveness of the fitted continuum with the reduced chi-square method. Ideally, should the reduced chi-square value increase or decrease when the polynomial fit increases degree? Like I fit a polynomial fit with n=1 up to n=10 so should I expect a decrease in the reduced chi-square values with increasing n</p>

<data-analysis><spectra><quasars>
Reduced chi square value
summer
# Reduced chi square value

(Asked by: username_0 on 2023-01-12)

I am currently plotting a continuum of observed data. I need to check the effectiveness of the fitted continuum with the reduced chi-square method. Ideally, should the reduced chi-square value increase or decrease when the polynomial fit increases degree? Like I fit a polynomial fit with n=1 up to n=10 so should I expect a decrease in the reduced chi-square values with increasing n

**Tags:** data-analysis, spectra, quasars


---

## 2 answers

### Answer by username_1 on 2023-01-12. Score: 3

The chi-squared should always decrease as you increase the complexity of the model (in this case by increasing the number of polynomial terms). The *reduced* chi-squared may decrease or increase, because the reduced chi-squared is the chi-squared divided by the number of degrees of freedom, but you have reduced the number of degrees of freedom.

Whether the additional complexity is warranted can be checked by comparing the change in the chi-squared against the number of degrees of freedom in the fit. For example you could use the [Bayesian Information Criterion (BIC)](https://en.wikipedia.org/wiki/Bayesian_information_criterion) but generally speaking, if the reduced chi-squared does not decrease then you are probably over-fitting the data.

> **Comments:**
> **username_2**: Another way to put the latter part of the final sentence is that the point at which the reduced chi-squared does not decrease significantly from the previous iteration with one fewer degree of freedom is the point at which one might well be overfitting.  
> **username_1**: @username_2 it can be more complicated. For example the true function could be odd or even in which case adding even or odd polynomial terms won't change the chi-squared and the reduced chi-squared will increase. But adding a further term could then see a significant decrease.  
> **username_2**: That is correct. Sometimes it's better to take a stepwise approach of looking at each model parameter individually and adding the one that results in the best improvement to the model, repeating until there are no modeling parameters left or the improvement is negligible. An alternative is to start with using all modeling parameters on the first round and then eliminating modeling parameters one by one based on some measure of statistical (in)significance, stopping the trimming when removing any one of the remaining parameters would trim too much (from a statistically significant perspective).  

---

### Answer by username_2 on 2023-01-13. Score: 1

> Should I expect a decrease in the reduced chi-square values with increasing n?

You should see a decrease up to the point where you start overfitting -- assuming that your assumption that the correct model is some kind of polynomial is correct. If it's not a polynomial, adding ever more terms might by luck capture ever more of the unmodeled variation.

Suppose one has captured data at $n>4$ points along some domain of interest, each with a bit of measurement error. Suppose the true model is a quadratic. The regression will improve with each step in going from a constant to a line to a quadratic to a cubic, so the chi-squared value will decrease with each step (but not much for the last step from a quadratic to a cubic), The reduced chi-squared value will decrease nicely on moving from a constant to a line to a quadratic. It won't reduce by much, or it might even increase on that last step from a quadratic to a cubic. That's a sign of overfitting (and also of using the wrong model).

Suppose one has lots of measurements (e.g., $n=1000$) and the measurement noise is not particularly small. Going from a constant to a line to a quadratic still results in a nice stepwise decrease in the reduced chi-square value. Here the step to a cubic might also result in a decrease because the addition of the cubic term might happen to capture some of the measurement noise and because dividing by 996 is not all that different from dividing by 997.


---

