  While supervised deep learning has achieved great success in a range of
applications, relatively little work has studied the discovery of knowledge
from unlabeled data. In this paper, we propose an unsupervised deep learning
framework to provide a potential solution for the problem that existing deep
learning techniques require large labeled data sets for completing the training
process. Our proposed introduces a new principle of joint learning on both deep
representations and GMM (Gaussian Mixture Model)-based deep modeling, and thus
an integrated objective function is proposed to facilitate the principle. In
comparison with the existing work in similar areas, our objective function has
two learning targets, which are created to be jointly optimized to achieve the
best possible unsupervised learning and knowledge discovery from unlabeled data
sets. While maximizing the first target enables the GMM to achieve the best
possible modeling of the data representations and each Gaussian component
corresponds to a compact cluster, maximizing the second term will enhance the
separability of the Gaussian components and hence the inter-cluster distances.
As a result, the compactness of clusters is significantly enhanced by reducing
the intra-cluster distances, and the separability is improved by increasing the
inter-cluster distances. Extensive experimental results show that the propose
method can improve the clustering performance compared with benchmark methods.
