  A common problem in Machine Learning and statistics consists in detecting
whether the current sample in a stream of data belongs to the same distribution
as previous ones, is an isolated outlier or inaugurates a new distribution of
data. We present a hierarchical Bayesian algorithm that aims at learning a
time-specific approximate posterior distribution of the parameters describing
the distribution of the data observed. We derive the update equations of the
variational parameters of the approximate posterior at each time step for
models from the exponential family, and show that these updates find
interesting correspondents in Reinforcement Learning (RL). In this perspective,
our model can be seen as a hierarchical RL algorithm that learns a posterior
distribution according to a certain stability confidence that is, in turn,
learned according to its own stability confidence. Finally, we show some
applications of our generic model, first in a RL context, next with an adaptive
Bayesian Autoregressive model, and finally in the context of Stochastic
Gradient Descent optimization.
