  Document classification is the detection specific content of interest in text
documents. In contrast to the data-driven machine learning classifiers,
knowledge-based classifiers can be constructed based on domain specific
knowledge, which usually takes the form of a collection of subject related
keywords. While typical knowledge-based classifiers compute a prediction score
based on the keyword abundance, it generally suffers from noisy detections due
to the lack of guiding principle in gauging the keyword matches. In this paper,
we propose a novel knowledge-based model equipped with Shannon Entropy, which
measures the richness of information and favors uniform and diverse keyword
matches. Without invoking any positive sample, such method provides a simple
and explainable solution for document classification. We show that the Shannon
Entropy significantly improves the recall at fixed level of false positive
rate. Also, we show that the model is more robust against change of data
distribution at inference while compared with traditional machine learning,
particularly when the positive training samples are very limited.
