  Anomaly detection in command shell sessions is a critical aspect of computer
security. Recent advances in deep learning and natural language processing,
particularly transformer-based models, have shown great promise for addressing
complex security challenges. In this paper, we implement a comprehensive
approach to detect anomalies in Unix shell sessions using a pretrained
DistilBERT model, leveraging both unsupervised and supervised learning
techniques to identify anomalous activity while minimizing data labeling. The
unsupervised method captures the underlying structure and syntax of Unix shell
commands, enabling the detection of session deviations from normal behavior.
Experiments on a large-scale enterprise dataset collected from production
systems demonstrate the effectiveness of our approach in detecting anomalous
behavior in Unix shell sessions. This work highlights the potential of
leveraging recent advances in transformers to address important computer
security challenges.
