  Automated machine learning techniques benefited from tremendous research
progress in recently. These developments and the continuous-growing demand for
machine learning experts led to the development of numerous AutoML tools.
However, these tools assume that the entire training dataset is available
upfront and that the underlying distribution does not change over time. These
assumptions do not hold in a data stream mining setting where an unbounded
stream of data cannot be stored and is likely to manifest concept drift.
Industry applications of machine learning on streaming data become more popular
due to the increasing adoption of real-time streaming patterns in IoT,
microservices architectures, web analytics, and other fields. The research
summarized in this paper surveys the state-of-the-art open-source AutoML tools,
applies them to data collected from streams, and measures how their performance
changes over time. For comparative purposes, batch, batch incremental and
instance incremental estimators are applied and compared. Moreover, a
meta-learning technique for online algorithm selection based on meta-feature
extraction is proposed and compared while model replacement and continual
AutoML techniques are discussed. The results show that off-the-shelf AutoML
tools can provide satisfactory results but in the presence of concept drift,
detection or adaptation techniques have to be applied to maintain the
predictive accuracy over time.
