  We consider how to effectively use prior knowledge when learning a Bayesian
model from streaming environments where the data come infinitely and
sequentially. This problem is highly important in the era of data explosion and
rich sources of precious external knowledge such as pre-trained models,
ontologies, Wikipedia, etc. We show that some existing approaches can forget
any knowledge very fast. We then propose a novel framework that enables to
incorporate the prior knowledge of different forms into a base Bayesian model
for data streams. Our framework subsumes some existing popular models for
time-series/dynamic data. Extensive experiments show that our framework
outperforms existing methods with a large margin. In particular, our framework
can help Bayesian models generalize well on extremely short text while other
methods overfit. The implementation of our framework is available at
https://github.com/bachtranxuan/TPS.git.
