  We revisit the method of mixture technique, also known as the Laplace method,
to study the concentration phenomenon in generic exponential families.
Combining the properties of Bregman divergence associated with log-partition
function of the family with the method of mixtures for super-martingales, we
establish a generic bound controlling the Bregman divergence between the
parameter of the family and a finite sample estimate of the parameter. Our
bound is time-uniform and makes appear a quantity extending the classical
information gain to exponential families, which we call the Bregman information
gain. For the practitioner, we instantiate this novel bound to several
classical families, e.g., Gaussian, Bernoulli, Exponential, Weibull, Pareto,
Poisson and Chi-square yielding explicit forms of the confidence sets and the
Bregman information gain. We further numerically compare the resulting
confidence bounds to state-of-the-art alternatives for time-uniform
concentration and show that this novel method yields competitive results.
Finally, we highlight the benefit of our concentration bounds on some
illustrative applications.
