  We show, using three empirical applications, that linear regression estimates
which rely on the assumption of sparsity are fragile in two ways. First, we
document that different choices of the regressor matrix that do not impact
ordinary least squares (OLS) estimates, such as the choice of baseline category
with categorical controls, can move sparsity-based estimates two standard
errors or more. Second, we develop two tests of the sparsity assumption based
on comparing sparsity-based estimators with OLS. The tests tend to reject the
sparsity assumption in all three applications. Unless the number of regressors
is comparable to or exceeds the sample size, OLS yields more robust results at
little efficiency cost.
