  We develop a data driven approach to perform clustering and end-to-end
feature learning simultaneously for streaming data that can adaptively detect
novel clusters in emerging data. Our approach, Adaptive Nonparametric
Variational Autoencoder (AdapVAE), learns the cluster membership through a
Bayesian Nonparametric (BNP) modeling framework with Deep Neural Networks
(DNNs) for feature learning. We develop a joint online variational inference
algorithm to learn feature representations and clustering assignments
simultaneously via iteratively optimizing the Evidence Lower Bound (ELBO). We
resolve the catastrophic forgetting \citep{kirkpatrick2017overcoming}
challenges with streaming data by adopting generative samples from the trained
AdapVAE using previous data, which avoids the need of storing and reusing past
data. We demonstrate the advantages of our model including adaptive novel
cluster detection without discarding useful information learned from past data,
high quality sample generation and comparable clustering performance as
end-to-end batch mode clustering methods on both image and text corpora
benchmark datasets.
