I would like to understand why, under the OLS model, the RSS (residual sum of squares) is distributed $$\chi^2\cdot (n-p)$$ ($p$ being the number of parameters in the model, $n$ the number of observations).

I apologize for asking such a basic question, but I seem to not be able to find the answer online (or in my, more application oriented, textbooks).

I consider the following linear model: ${y} = X \beta + \epsilon$.

The vector of residuals is estimated by

$$\hat{\epsilon} = y - X \hat{\beta} 
                     = (I - X (X'X)^{-1} X') y
                     = Q y
                     = Q (X \beta + \epsilon) = Q \epsilon$$

where $Q = I - X (X'X)^{-1} X'$.

Observe that $\textrm{tr}(Q) = n - p$ (the trace is invariant under cyclic permutation) and that $Q'=Q=Q^2$. The eigenvalues of $Q$ are therefore $0$ and $1$ (some details below). Hence, there exists a unitary matrix $V$ such that (matrices are diagonalizable by unitary matrices if and only if they are normal.)

$$V'QV = \Delta = \textrm{diag}(\underbrace{1, \ldots, 1}_{n-p \textrm{ times}}, \underbrace{0, \ldots, 0}_{p \textrm{ times}})$$

Now, let $K = V' \hat{\epsilon}$.

Since $\hat{\epsilon} \sim N(0, \sigma^2 Q)$, we have $K \sim N(0, \sigma^2 \Delta)$ and therefore $K_{n-p+1}=\ldots=K_n=0$. Thus

$$\frac{\|K\|^2}{\sigma^2} = \frac{\|K^{\star}\|^2}{\sigma^2} \sim \chi^2_{n-p}$$

with $K^{\star} = (K_1, \ldots, K_{n-p})'$.

Further, as $V$ is a unitary matrix, we also have

$$\|\hat{\epsilon}\|^2 = \|K\|^2=\|K^{\star}\|^2$$

Thus 

$$\frac{\textrm{RSS}}{\sigma^2} \sim \chi^2_{n-p}$$

Finally, observe that this result implies that 

$$E\left(\frac{\textrm{RSS}}{n-p}\right) = \sigma^2$$



Since $Q^2 - Q =0$, the minimal polynomial of $Q$ divides the polynomial $z^2 - z$. So, the eigenvalues of $Q$ are among $0$ and $1$. Since $\textrm{tr}(Q) = n-p$ is also the sum of the eigenvalues multiplied by their multiplicity, we necessarily have that $1$ is an eigenvalue with multiplicity $n-p$ and zero is an eigenvalue with multiplicity $p$.
