  Omitted variables are a common concern in empirical research. We distinguish
between two ways omitted variables can affect baseline estimates: by driving
them to zero or by reversing their sign. We show that, depending on how the
impact of omitted variables is measured, it can be substantially easier for
omitted variables to flip coefficient signs than to drive them to zero.
Consequently, results which are considered robust to being "explained away" by
omitted variables are not necessarily robust to sign changes. We show that this
behavior occurs with "Oster's delta" (Oster 2019), a commonly reported measure
of regression coefficient robustness to the presence of omitted variables.
Specifically, we show that any time this measure is large--suggesting that
omitted variables may be unimportant--a much smaller value reverses the sign of
the parameter of interest. Relatedly, we show that selection bias adjusted
estimands can be extremely sensitive to the choice of the sensitivity
parameter. Specifically, researchers commonly compute a bias adjustment under
the assumption that Oster's delta equals one. Under the alternative assumption
that delta is very close to one, but not exactly equal to one, we show that the
bias can instead be arbitrarily large. To address these concerns, we propose a
modified measure of robustness that accounts for such sign changes, and discuss
best practices for assessing sensitivity to omitted variables. We demonstrate
this sign flipping behavior in an empirical application to social capital and
the rise of the Nazi party, where we show how it can overturn conclusions about
robustness, and how our proposed modifications can be used to regain
robustness. We analyze three additional empirical applications as well. We
implement our proposed methods in the companion Stata module regsensitivity for
easy use in practice.
