  Recent advances in linguistic steganalysis have successively applied CNN,
RNN, GNN and other efficient deep models for detecting secret information in
generative texts. These methods tend to seek stronger feature extractors to
achieve higher steganalysis effects. However, we have found through experiments
that there actually exists significant difference between automatically
generated stego texts and carrier texts in terms of the conditional probability
distribution of individual words. Such kind of difference can be naturally
captured by the language model used for generating stego texts. Through further
experiments, we conclude that this ability can be transplanted to a text
classifier by pre-training and fine-tuning to improve the detection
performance. Motivated by this insight, we propose two methods for efficient
linguistic steganalysis. One is to pre-train a language model based on RNN, and
the other is to pre-train a sequence autoencoder. The results indicate that the
two methods have different degrees of performance gain compared to the randomly
initialized RNN, and the convergence speed is significantly accelerated.
Moreover, our methods achieved the best performance compared to related works,
while providing a solution for real-world scenario where there are more cover
texts than stego texts.
