<p>As the above comment pointed out, it is always good to provide exact definitions when asking a mathematical question.</p>
<p>But since your notation (or the notation in the book/lecture notes you are reading) is fairly standard, I'll fill in the definition from my interpretation and then give an answer.</p>
<p>In a generic multivariate regression <span class="math-container">$y = X \beta + e$</span>, we use <span class="math-container">$P_X$</span> to denote the projection matrix <span class="math-container">$P_X := X (X'X) ^{-1} X'$</span> and use <span class="math-container">$M_X := I - P_X$</span> to denote the annihilator matrix.</p>
<p>If we partition the big <span class="math-container">$X$</span> matrix into two parts as in <span class="math-container">$X = (X_1, X_2)$</span> and partition the big <span class="math-container">$\beta$</span> vector into <span class="math-container">$\beta = (\beta_1', \beta_2')'$</span> in the compatible way, then we can write <span class="math-container">$y = X_1 \beta_1 + X_2 \beta_2 + e$</span>.</p>
<p><strong>Theorem. The result we want to prove</strong> is that the projection matrix onto <span class="math-container">$X$</span> can be broken into two (orthogonal) parts: the projection matrix onto <span class="math-container">$X_1$</span> and the projection matrix onto <span class="math-container">$\tilde{X_2}$</span>, where <span class="math-container">$\tilde{X_2} := M_{X_1} X_2 = (I - P_{X_1}) X_2$</span> is the residual from the projection of <span class="math-container">${X_2}$</span> onto <span class="math-container">${X_1}$</span>.</p>
<p><strong>Algebraically, what we want to show is <span class="math-container">$P_X = P_{X_1} + P_{\tilde{X_2}}$</span>.</strong></p>
<p>Before we prove this, we note that this result is a type of the so-called <strong>Frisch-Waugh-Lovell theorem</strong> in econometrics (<a href="https://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem</a>).</p>
<p><strong>Proof.</strong> Using <span class="math-container">$X = (X_1, X_2)$</span>, we expand
<span class="math-container">$$
P_X = X (X'X) ^{-1} X' 
= (X_1, X_2) \left[\begin{pmatrix} X_1' \\ X_2' \end{pmatrix} (X_1, X_2)\right]^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix}
= (X_1, X_2) \begin{pmatrix} X_1'X_1 &amp; X_1'X_2 \\ X_2'X_1 &amp; X_2'X_2 \end{pmatrix}^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix}.
$$</span></p>
<p>We use the simplified notation
<span class="math-container">$$
A := \begin{pmatrix} A_{11} &amp; A_{12} \\ A_{21} &amp; A_{22} \end{pmatrix} 
:= \begin{pmatrix} X_1'X_1 &amp; X_1'X_2 \\ X_2'X_1 &amp; X_2'X_2 \end{pmatrix}
$$</span>
and also use
<span class="math-container">$$
A^{-1} := \begin{pmatrix} A^{11} &amp; A^{12} \\ A^{21} &amp; A^{22} \end{pmatrix} 
$$</span>
to denote the inverse of <span class="math-container">$A$</span>.</p>
<p>Now, using bock matrix inverse formula (<a href="https://en.wikipedia.org/wiki/Block_matrix" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Block_matrix</a>),   we can derive
<span class="math-container">$$
\begin{align}
A^{22} &amp; = A_{22.1}^{-1} := (A_{22} - A_{21} A_{11}^{-1} A_{12})^{-1} \\
&amp; =(X_2'X_2 - X_2'X_1 (X_1'X_1)^{-1} X_1'X_2)^{-1} \\
&amp; =(X_2' M_{X_1} X_2)^{-1} \\
&amp; =(\tilde{X_2}' \tilde{X_2})^{-1},\\
&amp; \\
A^{21} &amp; = - A^{22} A_{21} A_{11}^{-1} \\
&amp; = - (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1},\\
&amp; \\
A^{12} &amp; = A^{21\prime}, \\
&amp; \\
A^{11} &amp; = A_{11}^{-1} + A_{11}^{-1} A_{12} A^{22} A_{21} A_{11}^{-1} \\
&amp; = (X_1'X_1)^{-1} + (X_1'X_1)^{-1} X_1'X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1}.
\end{align}
$$</span></p>
<p>With these formulas, we go back to <span class="math-container">$P_X$</span> to derive
<span class="math-container">$$
\begin{align}
P_X &amp; = (X_1, X_2) \begin{pmatrix} X_1'X_1 &amp; X_1'X_2 \\ X_2'X_1 &amp; X_2'X_2 \end{pmatrix}^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix} \\
&amp; = (X_1, X_2) A^{-1} \begin{pmatrix} X_1' \\ X_2' \end{pmatrix} \\
&amp; = (X_1, X_2) \begin{pmatrix} A^{11} &amp; A^{12} \\ A^{21} &amp; A^{22} \end{pmatrix}  \begin{pmatrix} X_1' \\ X_2' \end{pmatrix} \\
&amp; = X_1 A^{11} X_1' + X_1 A^{12} X_2' + X_2 A^{21} X_1' + X_2 A^{22} X_2' \\
&amp; = X_1 (X_1'X_1)^{-1} X_1' + X_1 (X_1'X_1)^{-1} X_1'X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1} X_1' \\
&amp; - X_1 (X_1'X_1)^{-1} X_1'X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' \\
&amp; - X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2'X_1 (X_1'X_1)^{-1} X_1' \\
&amp; + X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' \\
&amp; = P_{X_1} + P_{X_1} X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' P_{X_1} \\
&amp; - P_{X_1} X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' - X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' P_{X_1} + X_2 (\tilde{X_2}' \tilde{X_2})^{-1} X_2' \\
&amp; = P_{X_1} + (X_2 - P_{X_1} X_2) (\tilde{X_2}' \tilde{X_2})^{-1} (X_2' - X_2' P_{X_1}) \\
&amp; = P_{X_1} + \tilde{X_2} (\tilde{X_2}' \tilde{X_2})^{-1} \tilde{X_2}' \\
&amp; = P_{X_1} + P_{\tilde{X_2}}. \\
\end{align}
$$</span>
Q.E.D.</p>
<p>P.S. Let me know if there is any typo. This is a lot of typing. :)</p>

CC BY-SA 4.0
2022-12-10T09:18:59.870Z
Thanks a lot for your detailed answer :) it really helps me to see full algebraical expression to understand what we really do. I tried to do something similar but I was lost. I appreciate your time.
Tatanik501
CC BY-SA 4.0
2022-12-09T10:44:53.287Z
2022-12-09T10:44:53.287Z
easyliving
<p>We have the below expression:
<span class="math-container">$$
P_{M_{1}.X_{2}} \cdot y
$$</span>
I guess its interpretation is that they are the <strong>fitted values obtained from the regression of</strong> <span class="math-container">$y$</span> <strong>on residuals that are obtained from the regression</strong> of <span class="math-container">$X_2$</span> on <span class="math-container">$X_1$</span>.</p>
<p>But what do we really mean by this? what is the thing we do actually with that regressions?</p>
<p>and How we can show that
<span class="math-container">$$
P_x - P_1 = P_{M_{1}.X_{2}}
$$</span></p>

CC BY-SA 4.0
2022-11-30T21:26:43.843Z
What does $x$ denote in $P_x$? And what is $P_1$? Is your model linear? Please give more details...
Bertrand
CC BY-SA 4.0
2022-12-01T06:45:32.933Z
$P_x$ vector of fitted values from the regression of $y = X_1 \cdot \beta_1   +X_2 \cdot \beta_2 + u$ and $P_1$ is the one for $y = X_1 \cdot \beta_1 + u_1$
Tatanik501
CC BY-SA 4.0
2022-12-09T08:43:31.303Z
@Tatanik501 Actually $P$ is the projection matrix which is a special case of a projection operator in the context of a finite dimensional subspace.
easyliving
<econometrics><regression>
how to use projection matrices in econometrics
Tatanik501
# how to use projection matrices in econometrics

(Asked by: username_0 on 2022-11-30)

We have the below expression:
$$
P\_{M\_{1}.X\_{2}} \cdot y
$$
I guess its interpretation is that they are the **fitted values obtained from the regression of** $y$ **on residuals that are obtained from the regression** of $X\_2$ on $X\_1$.

But what do we really mean by this? what is the thing we do actually with that regressions?

and How we can show that
$$
P\_x - P\_1 = P\_{M\_{1}.X\_{2}}
$$

**Tags:** econometrics, regression

> **Comments:**
> **username_1**: What does $x$ denote in $P_x$? And what is $P_1$? Is your model linear? Please give more details...  
> **username_0**: $P_x$ vector of fitted values from the regression of $y = X_1 \cdot \beta_1   +X_2 \cdot \beta_2 + u$ and $P_1$ is the one for $y = X_1 \cdot \beta_1 + u_1$  
> **username_2**: @username_0 Actually $P$ is the projection matrix which is a special case of a projection operator in the context of a finite dimensional subspace.  

---

## 1 answers

### Answer by username_2 on 2022-12-09. Score: 3 | Accepted answer

As the above comment pointed out, it is always good to provide exact definitions when asking a mathematical question.

But since your notation (or the notation in the book/lecture notes you are reading) is fairly standard, I'll fill in the definition from my interpretation and then give an answer.

In a generic multivariate regression $y = X \beta + e$, we use $P\_X$ to denote the projection matrix $P\_X := X (X'X) ^{-1} X'$ and use $M\_X := I - P\_X$ to denote the annihilator matrix.

If we partition the big $X$ matrix into two parts as in $X = (X\_1, X\_2)$ and partition the big $\beta$ vector into $\beta = (\beta\_1', \beta\_2')'$ in the compatible way, then we can write $y = X\_1 \beta\_1 + X\_2 \beta\_2 + e$.

**Theorem. The result we want to prove** is that the projection matrix onto $X$ can be broken into two (orthogonal) parts: the projection matrix onto $X\_1$ and the projection matrix onto $\tilde{X\_2}$, where $\tilde{X\_2} := M\_{X\_1} X\_2 = (I - P\_{X\_1}) X\_2$ is the residual from the projection of ${X\_2}$ onto ${X\_1}$.

**Algebraically, what we want to show is $P\_X = P\_{X\_1} + P\_{\tilde{X\_2}}$.**

Before we prove this, we note that this result is a type of the so-called **Frisch-Waugh-Lovell theorem** in econometrics (<https://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem>).

**Proof.** Using $X = (X\_1, X\_2)$, we expand
$$
P\_X = X (X'X) ^{-1} X'
= (X\_1, X\_2) \left[\begin{pmatrix} X\_1' \\ X\_2' \end{pmatrix} (X\_1, X\_2)\right]^{-1} \begin{pmatrix} X\_1' \\ X\_2' \end{pmatrix}
= (X\_1, X\_2) \begin{pmatrix} X\_1'X\_1 & X\_1'X\_2 \\ X\_2'X\_1 & X\_2'X\_2 \end{pmatrix}^{-1} \begin{pmatrix} X\_1' \\ X\_2' \end{pmatrix}.
$$

We use the simplified notation
$$
A := \begin{pmatrix} A\_{11} & A\_{12} \\ A\_{21} & A\_{22} \end{pmatrix}
:= \begin{pmatrix} X\_1'X\_1 & X\_1'X\_2 \\ X\_2'X\_1 & X\_2'X\_2 \end{pmatrix}
$$
and also use
$$
A^{-1} := \begin{pmatrix} A^{11} & A^{12} \\ A^{21} & A^{22} \end{pmatrix}
$$
to denote the inverse of $A$.

Now, using bock matrix inverse formula (<https://en.wikipedia.org/wiki/Block_matrix>), we can derive
$$
\begin{align}
A^{22} & = A\_{22.1}^{-1} := (A\_{22} - A\_{21} A\_{11}^{-1} A\_{12})^{-1} \\
& =(X\_2'X\_2 - X\_2'X\_1 (X\_1'X\_1)^{-1} X\_1'X\_2)^{-1} \\
& =(X\_2' M\_{X\_1} X\_2)^{-1} \\
& =(\tilde{X\_2}' \tilde{X\_2})^{-1},\\
& \\
A^{21} & = - A^{22} A\_{21} A\_{11}^{-1} \\
& = - (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2'X\_1 (X\_1'X\_1)^{-1},\\
& \\
A^{12} & = A^{21\prime}, \\
& \\
A^{11} & = A\_{11}^{-1} + A\_{11}^{-1} A\_{12} A^{22} A\_{21} A\_{11}^{-1} \\
& = (X\_1'X\_1)^{-1} + (X\_1'X\_1)^{-1} X\_1'X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2'X\_1 (X\_1'X\_1)^{-1}.
\end{align}
$$

With these formulas, we go back to $P\_X$ to derive
$$
\begin{align}
P\_X & = (X\_1, X\_2) \begin{pmatrix} X\_1'X\_1 & X\_1'X\_2 \\ X\_2'X\_1 & X\_2'X\_2 \end{pmatrix}^{-1} \begin{pmatrix} X\_1' \\ X\_2' \end{pmatrix} \\
& = (X\_1, X\_2) A^{-1} \begin{pmatrix} X\_1' \\ X\_2' \end{pmatrix} \\
& = (X\_1, X\_2) \begin{pmatrix} A^{11} & A^{12} \\ A^{21} & A^{22} \end{pmatrix} \begin{pmatrix} X\_1' \\ X\_2' \end{pmatrix} \\
& = X\_1 A^{11} X\_1' + X\_1 A^{12} X\_2' + X\_2 A^{21} X\_1' + X\_2 A^{22} X\_2' \\
& = X\_1 (X\_1'X\_1)^{-1} X\_1' + X\_1 (X\_1'X\_1)^{-1} X\_1'X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2'X\_1 (X\_1'X\_1)^{-1} X\_1' \\
& - X\_1 (X\_1'X\_1)^{-1} X\_1'X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2' \\
& - X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2'X\_1 (X\_1'X\_1)^{-1} X\_1' \\
& + X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2' \\
& = P\_{X\_1} + P\_{X\_1} X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2' P\_{X\_1} \\
& - P\_{X\_1} X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2' - X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2' P\_{X\_1} + X\_2 (\tilde{X\_2}' \tilde{X\_2})^{-1} X\_2' \\
& = P\_{X\_1} + (X\_2 - P\_{X\_1} X\_2) (\tilde{X\_2}' \tilde{X\_2})^{-1} (X\_2' - X\_2' P\_{X\_1}) \\
& = P\_{X\_1} + \tilde{X\_2} (\tilde{X\_2}' \tilde{X\_2})^{-1} \tilde{X\_2}' \\
& = P\_{X\_1} + P\_{\tilde{X\_2}}. \\
\end{align}
$$
Q.E.D.

P.S. Let me know if there is any typo. This is a lot of typing. :)

> **Comments:**
> **username_0**: Thanks a lot for your detailed answer :) it really helps me to see full algebraical expression to understand what we really do. I tried to do something similar but I was lost. I appreciate your time.  

---

