  In robust decision-making under non-Bayesian uncertainty, different robust
optimization criteria, such as maximin performance, minimax regret, and maximin
ratio, have been proposed. In many problems, all three criteria are
well-motivated and well-grounded from a decision-theoretic perspective, yet
different criteria give different prescriptions. This paper initiates a
systematic study of overfitting to robustness criteria. How good is a
prescription derived from one criterion when evaluated against another
criterion? Does there exist a prescription that performs well against all
criteria of interest? We formalize and study these questions through the
prototypical problem of robust pricing under various information structures,
including support, moments, and percentiles of the distribution of values. We
provide a unified analysis of three focal robust criteria across various
information structures and evaluate the relative performance of mechanisms
optimized for each criterion against the others. We find that mechanisms
optimized for one criterion often perform poorly against other criteria,
highlighting the risk of overfitting to a particular robustness criterion.
Remarkably, we show it is possible to design mechanisms that achieve good
performance across all three criteria simultaneously, suggesting that
decision-makers need not compromise among criteria.
