  This paper describes Galileo's performance in SemEval-2020 Task 12 on
detecting and categorizing offensive language in social media. For Offensive
Language Identification, we proposed a multi-lingual method using Pre-trained
Language Models, ERNIE and XLM-R. For offensive language categorization, we
proposed a knowledge distillation method trained on soft labels generated by
several supervised models. Our team participated in all three sub-tasks. In
Sub-task A - Offensive Language Identification, we ranked first in terms of
average F1 scores in all languages. We are also the only team which ranked
among the top three across all languages. We also took the first place in
Sub-task B - Automatic Categorization of Offense Types and Sub-task C - Offence
Target Identification.
