  We introduce LIMES, a new method for learning with non-stationary streaming
data, inspired by the recent success of meta-learning. The main idea is not to
attempt to learn a single classifier that would have to work well across all
occurring data distributions, nor many separate classifiers, but to exploit a
hybrid strategy: we learn a single set of model parameters from which a
specific classifier for any specific data distribution is derived via
classifier adaptation. Assuming a multi-class classification setting with
class-prior shift, the adaptation step can be performed analytically with only
the classifier's bias terms being affected. Another contribution of our work is
an extrapolation step that predicts suitable adaptation parameters for future
time steps based on the previous data. In combination, we obtain a lightweight
procedure for learning from streaming data with varying class distribution that
adds no trainable parameters and almost no memory or computational overhead
compared to training a single model. Experiments on a set of exemplary tasks
using Twitter data show that LIMES achieves higher accuracy than alternative
approaches, especially with respect to the relevant real-world metric of lowest
within-day accuracy.
