  Existing deep hierarchical topic models are able to extract semantically
meaningful topics from a text corpus in an unsupervised manner and
automatically organize them into a topic hierarchy. However, it is unclear how
to incorporate prior beliefs such as knowledge graph to guide the learning of
the topic hierarchy. To address this issue, we introduce TopicNet as a deep
hierarchical topic model that can inject prior structural knowledge as an
inductive bias to influence learning. TopicNet represents each topic as a
Gaussian-distributed embedding vector, projects the topics of all layers into a
shared embedding space, and explores both the symmetric and asymmetric
similarities between Gaussian embedding vectors to incorporate prior semantic
hierarchies. With an auto-encoding variational inference network, the model
parameters are optimized by minimizing the evidence lower bound and a
regularization term via stochastic gradient descent. Experiments on widely used
benchmarks show that TopicNet outperforms related deep topic models on
discovering deeper interpretable topics and mining better
document~representations.
