<p>When you add covariates, <span class="math-container">$\beta$</span> is interpreted as &quot;the effect of <span class="math-container">$D$</span> on <span class="math-container">$y$</span> holding fixed <span class="math-container">$X$</span>. Adding variables to <span class="math-container">$X$</span> can either increase or decrease the p-value depending on the relationship of <span class="math-container">$X, D,$</span> and <span class="math-container">$y$</span>.</p>

CC BY-SA 4.0
2021-09-20T13:16:08.857Z
2021-09-20T13:16:08.857Z
Michael Gmeiner
<p>First of all, let me tell you that it is bad statistics to compare <span class="math-container">$p$</span>-values across different specification. What you certainly should not do is to pick the specification based on the <span class="math-container">$p$</span>-values that you obtain.</p>
<p>Having said this. Assume that the right specification is given by:
<span class="math-container">$$
y_i = \alpha + \beta D_i + \gamma X_{i} + \varepsilon_i 
$$</span>
Assume you wrongly estimate:
<span class="math-container">$$
y_i = \alpha + \beta D_i + \delta_i
$$</span>
where now <span class="math-container">$\delta_i = \varepsilon_i + \gamma X_i$</span> captures both the random error <span class="math-container">$\varepsilon_i$</span> and the effect of <span class="math-container">$X_i$</span>.</p>
<p>If you would estimate the second specification, you actually identify the following:
<span class="math-container">$$
\begin{align*}
\hat \beta &amp;= \mathbb{E}(y_i|D_i = 1) - \mathbb{E}(y_i|D_i = 0),\\
&amp;= \beta + \mathbb{E}(\delta_i|D_i = 1) - \mathbb{E}(\delta_i|D_i = 0),\\
&amp;= \beta + \gamma\left[\mathbb{E}(X_i|D_i = 1) - \mathbb{E}(X_i|D_i = 0)\right]
\end{align*}
$$</span>
Depending on the second term, <span class="math-container">$\hat \beta$</span> might be lower or higher than <span class="math-container">$\beta$</span>. This will depend on the sign of <span class="math-container">$\gamma$</span> and the direction of correlation between <span class="math-container">$X_i$</span> and <span class="math-container">$D_i$</span>. For example if <span class="math-container">$\gamma &gt; 0$</span> and <span class="math-container">$D_i = 1$</span> is associated with higher values of <span class="math-container">$X_i$</span> then <span class="math-container">$\hat \beta &gt; \beta$</span>, so you will tend to overestimate the effect of <span class="math-container">$D_i$</span> on <span class="math-container">$y_i$</span> (as you are also capturing part of the effect of <span class="math-container">$X_i$</span> in your estimate).</p>
<p>Whether you should include <span class="math-container">$X_i$</span> or not into the regression is not something that can be answered using statistics alone. If <span class="math-container">$X_i$</span> captures confounders (i.e. something that affects both <span class="math-container">$y_i$</span> and <span class="math-container">$D_i$</span>), then yes you should add it as you otherwise do not capture the causal effect of <span class="math-container">$D_i$</span> on <span class="math-container">$y_i$</span> but also part of the effect of <span class="math-container">$X_i$</span> on <span class="math-container">$y_i$</span>.</p>

CC BY-SA 4.0
2021-09-20T13:26:37.443Z
2021-09-20T13:26:37.443Z
tdm
<p>Considering a Difference-in-Differences equation with a dummy variable:</p>
<p><span class="math-container">$$
y_i = \alpha + \beta D_i + Xit + \varepsilon_i.
$$</span>
While <span class="math-container">$$Xit$$</span>
is a set of covariates.
Today <strong>when I add more covariates, the p-value decrease</strong>. Is it normal?</p>
<p>And if it is a problem, what should I deal with that?</p>

CC BY-SA 4.0
2021-09-22T14:37:56.140Z
It can happen if $D$ and $X$ are not much correlated each other and X is important for explaining $y$, when X decreases the variance of the (remaining) error and the variability in D is not much affected by partialing out X, resulting in a reduced standard error.
chan1142
<econometrics>
Is there any case that the coefficient of variable of interest in DiD more significant when adding more variables?
Phil Nguyen
# Is there any case that the coefficient of variable of interest in DiD more significant when adding more variables?

(Asked by: username_0 on 2021-09-20)

Considering a Difference-in-Differences equation with a dummy variable:

$$
y\_i = \alpha + \beta D\_i + Xit + \varepsilon\_i.
$$
While $$Xit$$
is a set of covariates.
Today **when I add more covariates, the p-value decrease**. Is it normal?

And if it is a problem, what should I deal with that?

**Tags:** econometrics

> **Comments:**
> **username_1**: It can happen if $D$ and $X$ are not much correlated each other and X is important for explaining $y$, when X decreases the variance of the (remaining) error and the variability in D is not much affected by partialing out X, resulting in a reduced standard error.  

---

## 2 answers

### Answer by username_3 on 2021-09-20. Score: 3 | Accepted answer

First of all, let me tell you that it is bad statistics to compare $p$-values across different specification. What you certainly should not do is to pick the specification based on the $p$-values that you obtain.

Having said this. Assume that the right specification is given by:
$$
y\_i = \alpha + \beta D\_i + \gamma X\_{i} + \varepsilon\_i
$$
Assume you wrongly estimate:
$$
y\_i = \alpha + \beta D\_i + \delta\_i
$$
where now $\delta\_i = \varepsilon\_i + \gamma X\_i$ captures both the random error $\varepsilon\_i$ and the effect of $X\_i$.

If you would estimate the second specification, you actually identify the following:
$$
\begin{align\*}
\hat \beta &= \mathbb{E}(y\_i|D\_i = 1) - \mathbb{E}(y\_i|D\_i = 0),\\
&= \beta + \mathbb{E}(\delta\_i|D\_i = 1) - \mathbb{E}(\delta\_i|D\_i = 0),\\
&= \beta + \gamma\left[\mathbb{E}(X\_i|D\_i = 1) - \mathbb{E}(X\_i|D\_i = 0)\right]
\end{align\*}
$$
Depending on the second term, $\hat \beta$ might be lower or higher than $\beta$. This will depend on the sign of $\gamma$ and the direction of correlation between $X\_i$ and $D\_i$. For example if $\gamma > 0$ and $D\_i = 1$ is associated with higher values of $X\_i$ then $\hat \beta > \beta$, so you will tend to overestimate the effect of $D\_i$ on $y\_i$ (as you are also capturing part of the effect of $X\_i$ in your estimate).

Whether you should include $X\_i$ or not into the regression is not something that can be answered using statistics alone. If $X\_i$ captures confounders (i.e. something that affects both $y\_i$ and $D\_i$), then yes you should add it as you otherwise do not capture the causal effect of $D\_i$ on $y\_i$ but also part of the effect of $X\_i$ on $y\_i$.


---

### Answer by username_2 on 2021-09-20. Score: 2

When you add covariates, $\beta$ is interpreted as "the effect of $D$ on $y$ holding fixed $X$. Adding variables to $X$ can either increase or decrease the p-value depending on the relationship of $X, D,$ and $y$.


---

