  Tail averaging consists in averaging the last examples in a stream. Common
techniques either have a memory requirement which grows with the number of
samples to average, are not available at every timestep or do not accomodate
growing windows. We propose two techniques with a low constant memory cost that
perform tail averaging with access to the average at every time step. We also
show how one can improve the accuracy of that average at the cost of increased
memory consumption.
