  We derive tight and computable bounds on the bias of statistical estimators,
or more generally of quantities of interest, when evaluated on a baseline model
P rather than on the typically unknown true model Q. Our proposed method
combines the scalable information inequality derived by P. Dupuis, K.Chowdhary,
the authors and their collaborators together with classical concentration
inequalities (such as Bennett's and Hoeffding-Azuma inequalities). Our bounds
are expressed in terms of the Kullback-Leibler divergence R(Q||P) of model Q
with respect to P and the moment generating function for the statistical
estimator under P. Furthermore, concentration inequalities, i.e. bounds on
moment generating functions, provide tight and computationally inexpensive
model bias bounds for quantities of interest. Finally, they allow us to derive
rigorous confidence bands for statistical estimators that account for model
bias and are valid for an arbitrary amount of data.
