  Continual Learning (CL) often relies on the availability of extensive
annotated datasets, an assumption that is unrealistically time-consuming and
costly in practice. We explore a novel paradigm termed name-only continual
learning where time and cost constraints prohibit manual annotation. In this
scenario, learners adapt to new category shifts using only category names
without the luxury of annotated training data. Our proposed solution leverages
the expansive and ever-evolving internet to query and download uncurated
webly-supervised data for image classification. We investigate the reliability
of our web data and find them comparable, and in some cases superior, to
manually annotated datasets. Additionally, we show that by harnessing the web,
we can create support sets that surpass state-of-the-art name-only
classification that create support sets using generative models or image
retrieval from LAION-5B, achieving up to 25% boost in accuracy. When applied
across varied continual learning contexts, our method consistently exhibits a
small performance gap in comparison to models trained on manually annotated
datasets. We present EvoTrends, a class-incremental dataset made from the web
to capture real-world trends, created in just minutes. Overall, this paper
underscores the potential of using uncurated webly-supervised data to mitigate
the challenges associated with manual data labeling in continual learning.
