<p>Consider the restricted least squares problem:
<span class="math-container">$$
\min \underbrace{(Y - Xb_\ast)'}_{(1\times n)}\underbrace{(Y - Xb_\ast)}_{n\times 1} \text{ s.t. } \underbrace{R}_{(j \times k)}\underbrace{b_\ast}_{(k \times 1)} - \underbrace{q}_{(j \times 1)}
$$</span>
We set up the Lagrangian
<span class="math-container">$$
L(b_\ast,\lambda) = (Y - Xb_\ast)'(Y - Xb_\ast) - 2 \underbrace{\lambda'}_{(1 \times j)}(Rb_\ast-q).
$$</span>
The first order conditions give:
<span class="math-container">$$
\underbrace{X'(Y - X b_\ast)}_{(k \times k)} + \underbrace{R'\lambda}_{(k \times k)} = 0.
$$</span>
This gives:
<span class="math-container">$$
b_\ast = \underbrace{(X'X)^{-1}X'Y}_{b} - (X'X)^{-1}R'\lambda.
$$</span>
The first term is the usual (unrestricted) OLS estimator, say <span class="math-container">$b$</span>. Multiplying both sides with <span class="math-container">$R$</span> gives:
<span class="math-container">$$
\underbrace{Rb_\ast}_{q} = \underbrace{R b}_{(j \times 1)} - \underbrace{R(X'X)^{-1}R'}_{(j \times j)} \underbrace{\lambda}_{j \times j}
$$</span>
If <span class="math-container">$R(X'X)^{-1}R'$</span> is of full rank, this can be solved for <span class="math-container">$\lambda$</span>:
<span class="math-container">$$
\lambda = (R(X'X)^{-1}R')^{-1}(Rb - q).
$$</span>
Subsituting into the expression for <span class="math-container">$b_\ast$</span> gives:
<span class="math-container">$$
b_\ast - b = \underbrace{(X'X)^{-1}}_{(k \times k)} \underbrace{R'}_{(k \times j)}\underbrace{(R(X'X)^{-1}R')^{-1}}_{(j \times j)} \underbrace{(q - Rb)}_{(j \times 1)}
$$</span></p>
<p>Now, substitute this into the expression <span class="math-container">$(b-b_\ast)'X'X(b - b_ast)$</span></p>
<p><span class="math-container">$$
(q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} X'X (X'X)^{-1}R'(R(X'X)^{-1}R')^{-1}(q - Rb)
$$</span></p>
<p>We can simplify the middle part as <span class="math-container">$X'X (X'X)^{-1} = I$</span>.
<span class="math-container">$$
(q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} R'(R(X'X)^{-1}R')^{-1}(q - Rb)
$$</span>
Next, also <span class="math-container">$R(X'X)^{-1}R'(R(X'X)^{-1}R')^{-1} = I$</span> so we get that:
<span class="math-container">$$
(b- b_\ast) X'X (b - b_\ast) = (q - Rb)' (R(X'X)^{-1}R')^{-1}(q - Rb)
$$</span>
This is expression (2).</p>

CC BY-SA 4.0
2023-11-03T08:30:54.837Z
2023-11-03T08:30:54.837Z
tdm
<p>In Greene's <em>Econometric Analysis</em> there is a derivation regarding the F stat. The setup is a null hypothesis of the form:  <span class="math-container">$H_0: R\beta =q$</span> where <span class="math-container">$\beta$</span> is a <span class="math-container">$k\times 1$</span> vector of parameters, <span class="math-container">$R$</span> is a <span class="math-container">$J\times k$</span> matrix and <span class="math-container">$q$</span> is a <span class="math-container">$J\times 1$</span> vector. <span class="math-container">$R$</span> and <span class="math-container">$q$</span> together define restrictions that are hypothesized. <span class="math-container">$e_*$</span> are residuals that result from OLS when imposing the restrictions and <span class="math-container">$e$</span> are residuals when estimating OLS without restrictions. <span class="math-container">$b_*$</span> and <span class="math-container">$b$</span> are respective parameter estimates. (Note that <span class="math-container">$Rb_*=q$</span>).</p>
<p>Greene's derivation has a portion in which this line is shown,</p>
<p><span class="math-container">$$(1)  \hspace{3.2cm}    e_*'e_*=e'e +(b-b_*)'X'X(b-b_*) $$</span>
The next line shows,</p>
<p><span class="math-container">$$(2)  \hspace{0.5cm}e_*'e_*-e'e =(Rb-q)'[R(X'X)^{-1}R']^{-1}(Rb-q) $$</span></p>
<p>I am curious how to go between these lines. If <span class="math-container">$R$</span> were an invertible square matrix, it is immediate because <span class="math-container">$e'e +(b-b_*)'X'X(b-b_*) =e'e +(b-b_*)'R'(R')^{-1}X'XR^{-1}R(b-b_*)$</span> then we simplify further.</p>
<p><span class="math-container">$R$</span> is not an invertible square matrix however, so this can't be the method.
Futhermore, appealing to the <a href="https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse" rel="noreferrer">pseudoinverse</a> does not help, because <span class="math-container">$R$</span> does not have full column rank, thus <span class="math-container">$R^+R\ne I$</span> and we can't proceed with <span class="math-container">$e'e +(b-b_*)'X'X(b-b_*) =e'e +(b-b_*)'R'(R')^{+}X'XR^{+}R(b-b_*)$</span>.</p>
<p>In short, there must be an algebraic trick to go from equation (1) to (2). I am curious what that trick is.</p>

CC BY-SA 4.0
2023-10-18T16:14:31.313Z
This looks like a good candidate for [Cross Validated](https://stats.stackexchange.com/questions?tab=Newest) Stack Exchange.
Richard Hardy
CC BY-SA 4.0
2023-10-19T05:30:07.983Z
Michael: Unfortunately, the proof at the link below uses different notation that you used but I'm quite confident that you can follow it and map it to your case. I did find a textbook derivation that uses your notation ( a introduction to econometric theory by james davidson ) but it's a fairly new text and probably not worth purchasing for one derivation. http://web.vu.lt/mif/a.buteikis/wp-content/uploads/PE_Book/4-4-Multiple-RLS.html
mark leeds
CC BY-SA 4.0
2023-10-20T08:54:08.967Z
Mark Leeds this is great. Thank you. You could consider making this an answer rather than a comment.
Michael Gmeiner
CC BY-SA 4.0
2023-10-20T16:23:51.917Z
Hi Michael: I'm glad it helps. I'm running out now but I'll try to remember later. Note that I re-read my comment in above and I don't mean to denigrate Davidson's book. It's quite a nice intro book. If someone is interested in that type of book, I highly recommend it.
mark leeds
<econometrics><hypothesis-testing><f-test>
Portion of Derivation of F-test
Michael Gmeiner
# Portion of Derivation of F-test

(Asked by: username_0 on 2023-10-17)

In Greene's *Econometric Analysis* there is a derivation regarding the F stat. The setup is a null hypothesis of the form: $H\_0: R\beta =q$ where $\beta$ is a $k\times 1$ vector of parameters, $R$ is a $J\times k$ matrix and $q$ is a $J\times 1$ vector. $R$ and $q$ together define restrictions that are hypothesized. $e\_\*$ are residuals that result from OLS when imposing the restrictions and $e$ are residuals when estimating OLS without restrictions. $b\_\*$ and $b$ are respective parameter estimates. (Note that $Rb\_\*=q$).

Greene's derivation has a portion in which this line is shown,

$$(1) \hspace{3.2cm} e\_\*'e\_\*=e'e +(b-b\_\*)'X'X(b-b\_\*) $$
The next line shows,

$$(2) \hspace{0.5cm}e\_\*'e\_\*-e'e =(Rb-q)'[R(X'X)^{-1}R']^{-1}(Rb-q) $$

I am curious how to go between these lines. If $R$ were an invertible square matrix, it is immediate because $e'e +(b-b\_\*)'X'X(b-b\_\*) =e'e +(b-b\_\*)'R'(R')^{-1}X'XR^{-1}R(b-b\_\*)$ then we simplify further.

$R$ is not an invertible square matrix however, so this can't be the method.
Futhermore, appealing to the [pseudoinverse](https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse) does not help, because $R$ does not have full column rank, thus $R^+R\ne I$ and we can't proceed with $e'e +(b-b\_\*)'X'X(b-b\_\*) =e'e +(b-b\_\*)'R'(R')^{+}X'XR^{+}R(b-b\_\*)$.

In short, there must be an algebraic trick to go from equation (1) to (2). I am curious what that trick is.

**Tags:** econometrics, hypothesis-testing, f-test

> **Comments:**
> **username_1**: This looks like a good candidate for [Cross Validated](https://stats.stackexchange.com/questions?tab=Newest) Stack Exchange.  
> **username_2**: Michael: Unfortunately, the proof at the link below uses different notation that you used but I'm quite confident that you can follow it and map it to your case. I did find a textbook derivation that uses your notation ( a introduction to econometric theory by james davidson ) but it's a fairly new text and probably not worth purchasing for one derivation. http://web.vu.lt/mif/a.buteikis/wp-content/uploads/PE_Book/4-4-Multiple-RLS.html  
> **username_0**: Mark Leeds this is great. Thank you. You could consider making this an answer rather than a comment.  
> **username_2**: Hi Michael: I'm glad it helps. I'm running out now but I'll try to remember later. Note that I re-read my comment in above and I don't mean to denigrate Davidson's book. It's quite a nice intro book. If someone is interested in that type of book, I highly recommend it.  

---

## 1 answers

### Answer by username_3 on 2023-11-03. Score: 4 | Accepted answer

Consider the restricted least squares problem:
$$
\min \underbrace{(Y - Xb\_\ast)'}\_{(1\times n)}\underbrace{(Y - Xb\_\ast)}\_{n\times 1} \text{ s.t. } \underbrace{R}\_{(j \times k)}\underbrace{b\_\ast}\_{(k \times 1)} - \underbrace{q}\_{(j \times 1)}
$$
We set up the Lagrangian
$$
L(b\_\ast,\lambda) = (Y - Xb\_\ast)'(Y - Xb\_\ast) - 2 \underbrace{\lambda'}\_{(1 \times j)}(Rb\_\ast-q).
$$
The first order conditions give:
$$
\underbrace{X'(Y - X b\_\ast)}\_{(k \times k)} + \underbrace{R'\lambda}\_{(k \times k)} = 0.
$$
This gives:
$$
b\_\ast = \underbrace{(X'X)^{-1}X'Y}\_{b} - (X'X)^{-1}R'\lambda.
$$
The first term is the usual (unrestricted) OLS estimator, say $b$. Multiplying both sides with $R$ gives:
$$
\underbrace{Rb\_\ast}\_{q} = \underbrace{R b}\_{(j \times 1)} - \underbrace{R(X'X)^{-1}R'}\_{(j \times j)} \underbrace{\lambda}\_{j \times j}
$$
If $R(X'X)^{-1}R'$ is of full rank, this can be solved for $\lambda$:
$$
\lambda = (R(X'X)^{-1}R')^{-1}(Rb - q).
$$
Subsituting into the expression for $b\_\ast$ gives:
$$
b\_\ast - b = \underbrace{(X'X)^{-1}}\_{(k \times k)} \underbrace{R'}\_{(k \times j)}\underbrace{(R(X'X)^{-1}R')^{-1}}\_{(j \times j)} \underbrace{(q - Rb)}\_{(j \times 1)}
$$

Now, substitute this into the expression $(b-b\_\ast)'X'X(b - b\_ast)$

$$
(q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} X'X (X'X)^{-1}R'(R(X'X)^{-1}R')^{-1}(q - Rb)
$$

We can simplify the middle part as $X'X (X'X)^{-1} = I$.
$$
(q - Rb)' (R(X'X)^{-1}R')^{-1}R(X'X)^{-1} R'(R(X'X)^{-1}R')^{-1}(q - Rb)
$$
Next, also $R(X'X)^{-1}R'(R(X'X)^{-1}R')^{-1} = I$ so we get that:
$$
(b- b\_\ast) X'X (b - b\_\ast) = (q - Rb)' (R(X'X)^{-1}R')^{-1}(q - Rb)
$$
This is expression (2).


---

