  We analyze different types of simulations that applied researchers may use to
assess their inference methods. We show that different types of simulations
vary in many dimensions when considered as inference assessments. Moreover, we
show that natural ways of running simulations may lead to misleading
conclusions, and we propose alternatives. We then provide evidence that even
some simple assessments can detect problems in many different settings.
Alternative assessments that potentially better approximate the true data
generating process may detect problems that simpler assessments would not
detect. However, they are not uniformly dominant in this dimension, and may
imply some costs.
