  This dissertation presents an evaluation of several language models on
software defect datasets. A language Model (LM) "can provide word
representation and probability indication of word sequences as the core
component of an NLP system." Language models for source code are specified for
tasks in the software engineering field. While some models are directly the NLP
ones, others contain structural information that is uniquely owned by source
code. Software defects are defects in the source code that lead to unexpected
behaviours and malfunctions at all levels. This study provides an original
attempt to detect these defects at three different levels (syntactical,
algorithmic and general) We also provide a tool chain that researchers can use
to reproduce the experiments. We have tested the different models against
different datasets, and performed an analysis over the results. Our original
attempt to deploy bert, the state-of-the-art model for multitasks, leveled or
outscored all other models compared.
