  Trustworthy machine learning aims at combating distributional uncertainties
in training data distributions compared to population distributions. Typical
treatment frameworks include the Bayesian approach, (min-max) distributionally
robust optimization (DRO), and regularization. However, two issues have to be
raised: 1) All these methods are biased estimators of the true optimal cost; 2)
the prior distribution in the Bayesian method, the radius of the distributional
ball in the DRO method, and the regularizer in the regularization method are
difficult to specify. This paper studies a new framework that unifies the three
approaches and that addresses the two challenges mentioned above. The
asymptotic properties (e.g., consistency and asymptotic normalities),
non-asymptotic properties (e.g., unbiasedness and generalization error bound),
and a Monte--Carlo-based solution method of the proposed model are studied. The
new model reveals the trade-off between the robustness to the unseen data and
the specificity to the training data.
