  The goal of this paper is to explore the basic Approximate Bayesian
Computation (ABC) algorithm via the lens of information theory. ABC is a widely
used algorithm in cases where the likelihood of the data is hard to work with
or intractable, but one can simulate from it. We use relative entropy ideas to
analyze the behavior of the algorithm as a function of the threshold parameter
and of the size of the data. Relative entropy here is data driven as it depends
on the values of the observed statistics. Relative entropy also allows us to
explore the effect of the distance metric and sets up a mathematical framework
for sensitivity analysis allowing to find important directions which could lead
to lower computational cost of the algorithm for the same level of accuracy. In
addition, we also investigate the bias of the estimators for generic
observables as a function of both the threshold parameters and the size of the
data. Our analysis provides error bounds on performance for positive tolerances
and finite sample sizes. Simulation studies complement and illustrate the
theoretical results.
