  Adapting to drifting data streams is a significant challenge in online
learning. Concept drift must be detected for effective model adaptation to
evolving data properties. Concept drift can impact the data distribution
entirely or partially, which makes it difficult for drift detectors to
accurately identify the concept drift. Despite the numerous concept drift
detectors in the literature, standardized procedures and benchmarks for
comprehensive evaluation considering the locality of the drift are lacking. We
present a novel categorization of concept drift based on its locality and
scale. A systematic approach leads to a set of 2,760 benchmark problems,
reflecting various difficulty levels following our proposed categorization. We
conduct a comparative assessment of 9 state-of-the-art drift detectors across
diverse difficulties, highlighting their strengths and weaknesses for future
research. We examine how drift locality influences the classifier performance
and propose strategies for different drift categories to minimize the recovery
time. Lastly, we provide lessons learned and recommendations for future concept
drift research. Our benchmark data streams and experiments are publicly
available at https://github.com/gabrieljaguiar/locality-concept-drift.
