  We study the neural-linear bandit model for solving sequential
decision-making problems with high dimensional side information. Neural-linear
bandits leverage the representation power of deep neural networks and combine
it with efficient exploration mechanisms, designed for linear contextual
bandits, on top of the last hidden layer. Since the representation is being
optimized during learning, information regarding exploration with "old"
features is lost. Here, we propose the first limited memory neural-linear
bandit that is resilient to this phenomenon, which we term catastrophic
forgetting. We evaluate our method on a variety of real-world data sets,
including regression, classification, and sentiment analysis, and observe that
our algorithm is resilient to catastrophic forgetting and achieves superior
performance.
