  The recent breakthroughs in computer vision have benefited from the
availability of large representative datasets (e.g. ImageNet and COCO) for
training. Yet, robotic vision poses unique challenges for applying visual
algorithms developed from these standard computer vision datasets due to their
implicit assumption over non-varying distributions for a fixed set of tasks.
Fully retraining models each time a new task becomes available is infeasible
due to computational, storage and sometimes privacy issues, while na\"{i}ve
incremental strategies have been shown to suffer from catastrophic forgetting.
It is crucial for the robots to operate continuously under open-set and
detrimental conditions with adaptive visual perceptual systems, where lifelong
learning is a fundamental capability. However, very few datasets and benchmarks
are available to evaluate and compare emerging techniques. To fill this gap, we
provide a new lifelong robotic vision dataset ("OpenLORIS-Object") collected
via RGB-D cameras. The dataset embeds the challenges faced by a robot in the
real-life application and provides new benchmarks for validating lifelong
object recognition algorithms. Moreover, we have provided a testbed of $9$
state-of-the-art lifelong learning algorithms. Each of them involves $48$ tasks
with $4$ evaluation metrics over the OpenLORIS-Object dataset. The results
demonstrate that the object recognition task in the ever-changing difficulty
environments is far from being solved and the bottlenecks are at the
forward/backward transfer designs. Our dataset and benchmark are publicly
available at at
\href{https://lifelong-robotic-vision.github.io/dataset/object}{\underline{https://lifelong-robotic-vision.github.io/dataset/object}}.
