  Moderation of reader comments is a significant problem for online news
platforms. Here, we experiment with models for automatic moderation, using a
dataset of comments from a popular Croatian newspaper. Our analysis shows that
while comments that violate the moderation rules mostly share common linguistic
and thematic features, their content varies across the different sections of
the newspaper. We therefore make our models topic-aware, incorporating semantic
features from a topic model into the classification decision. Our results show
that topic information improves the performance of the model, increases its
confidence in correct outputs, and helps us understand the model's outputs.
