  Due to the rising number of sophisticated customer functionalities,
electronic control units (ECUs) are increasingly integrated into modern
automotive systems. However, the high connectivity between the in-vehicle and
the external networks paves the way for hackers who could exploit in-vehicle
network protocols' vulnerabilities. Among these protocols, the Controller Area
Network (CAN), known as the most widely used in-vehicle networking technology,
lacks encryption and authentication mechanisms, making the communications
delivered by distributed ECUs insecure. Inspired by the outstanding performance
of bidirectional encoder representations from transformers (BERT) for improving
many natural language processing tasks, we propose in this paper ``CAN-BERT", a
deep learning based network intrusion detection system, to detect cyber attacks
on CAN bus protocol. We show that the BERT model can learn the sequence of
arbitration identifiers (IDs) in the CAN bus for anomaly detection using the
``masked language model" unsupervised training objective. The experimental
results on the ``Car Hacking: Attack \& Defense Challenge 2020" dataset show
that ``CAN-BERT" outperforms state-of-the-art approaches. In addition to being
able to identify in-vehicle intrusions in real-time within 0.8 ms to 3 ms w.r.t
CAN ID sequence length, it can also detect a wide variety of cyberattacks with
an F1-score of between 0.81 and 0.99.
