  Incremental learning is useful if an AI agent needs to integrate data from a
stream. The problem is non trivial if the agent runs on a limited computational
budget and has a bounded memory of past data. In a deep learning approach, the
constant computational budget requires the use of a fixed architecture for all
incremental states. The bounded memory generates data imbalance in favor of new
classes and a prediction bias toward them appears. This bias is commonly
countered by introducing a data balancing step in addition to the basic network
training. We depart from this approach and propose simple but efficient scaling
of past class classifier weights to make them more comparable to those of new
classes. Scaling exploits incremental state level statistics and is applied to
the classifiers learned in the initial state of classes in order to profit from
all their available data. We also question the utility of the widely used
distillation loss component of incremental learning algorithms by comparing it
to vanilla fine tuning in presence of a bounded memory. Evaluation is done
against competitive baselines using four public datasets. Results show that the
classifier weights scaling and the removal of the distillation are both
beneficial.
