  We present the One Pass ImageNet (OPIN) problem, which aims to study the
effectiveness of deep learning in a streaming setting. ImageNet is a widely
known benchmark dataset that has helped drive and evaluate recent advancements
in deep learning. Typically, deep learning methods are trained on static data
that the models have random access to, using multiple passes over the dataset
with a random shuffle at each epoch of training. Such data access assumption
does not hold in many real-world scenarios where massive data is collected from
a stream and storing and accessing all the data becomes impractical due to
storage costs and privacy concerns. For OPIN, we treat the ImageNet data as
arriving sequentially, and there is limited memory budget to store a small
subset of the data. We observe that training a deep network in a single pass
with the same training settings used for multi-epoch training results in a huge
drop in prediction accuracy. We show that the performance gap can be
significantly decreased by paying a small memory cost and utilizing techniques
developed for continual learning, despite the fact that OPIN differs from
typical continual problem settings. We propose using OPIN to study
resource-efficient deep learning.
