  A deep clustering network is desired for data streams because of its aptitude
in extracting natural features thus bypassing the laborious feature engineering
step. While automatic construction of the deep networks in streaming
environments remains an open issue, it is also hindered by the expensive
labeling cost of data streams rendering the increasing demand for unsupervised
approaches. This paper presents an unsupervised approach of deep clustering
network construction on the fly via simultaneous deep learning and clustering
termed Autonomous Deep Clustering Network (ADCN). It combines the feature
extraction layer and autonomous fully connected layer in which both network
width and depth are self-evolved from data streams based on the bias-variance
decomposition of reconstruction loss. The self-clustering mechanism is
performed in the deep embedding space of every fully connected layer while the
final output is inferred via the summation of cluster prediction score.
Further, a latent-based regularization is incorporated to resolve the
catastrophic forgetting issue. A rigorous numerical study has shown that ADCN
produces better performance compared to its counterparts while offering fully
autonomous construction of ADCN structure in streaming environments with the
absence of any labeled samples for model updates. To support the reproducible
research initiative, codes, supplementary material, and raw results of ADCN are
made available in \url{https://tinyurl.com/AutonomousDCN}.
