  In this study, we propose to evaluate the use of deep learning methods for
semantic classification at the sentence level to accelerate the process of
corpus building in the field of humanities and linguistics, a traditional and
time-consuming task. We introduce a novel corpus comprising around 2500
sentences spanning from 300 BCE to 900 CE including sexual semantics (medical,
erotica, etc.). We evaluate various sentence classification approaches and
different input embedding layers, and show that all consistently outperform
simple token-based searches. We explore the integration of idiolectal and
sociolectal metadata embeddings (centuries, author, type of writing), but find
that it leads to overfitting. Our results demonstrate the effectiveness of this
approach, achieving high precision and true positive rates (TPR) of
respectively 70.60% and 86.33% using HAN. We evaluate the impact of the dataset
size on the model performances (420 instead of 2013), and show that, while our
models perform worse, they still offer a high enough precision and TPR, even
without MLM, respectively 69% and 51%. Given the result, we provide an analysis
of the attention mechanism as a supporting added value for humanists in order
to produce more data.
