  Recent industry reports assure the rise of web robots which comprise more
than half of the total web traffic. They not only threaten the security,
privacy and efficiency of the web but they also distort analytics and metrics,
doubting the veracity of the information being promoted. In the academic
publishing domain, this can cause articles to be faulty presented as prominent
and influential. In this paper, we present our approach on detecting web robots
in academic publishing websites. We use different supervised learning
algorithms with a variety of characteristics deriving from both the log files
of the server and the content served by the website. Our approach relies on the
assumption that human users will be interested in specific domains or articles,
while web robots crawl a web library incoherently. We experiment with features
adopted in previous studies with the addition of novel semantic characteristics
which derive after performing a semantic analysis using the Latent Dirichlet
Allocation (LDA) algorithm. Our real-world case study shows promising results,
pinpointing the significance of semantic features in the web robot detection
problem.
