  Naive Bayes is a popular probabilistic model appreciated for its simplicity
and interpretability. However, the usual form of the related classifier suffers
from two major problems. First, as caring about the observations' law, it
cannot consider complex features. Moreover, it considers the conditional
independence of the observations given the hidden variable. This paper
introduces the original Neural Naive Bayes, modeling the parameters of the
classifier induced from the Naive Bayes with neural network functions. This
allows to correct the first problem. We also introduce new Neural Pooled Markov
Chain models, alleviating the independence condition. We empirically study the
benefits of these models for Sentiment Analysis, dividing the error rate of the
usual classifier by 4.5 on the IMDB dataset with the FastText embedding.
