  Toxicity is a prevalent social behavior that involves the use of hate speech,
offensive language, bullying, and abusive speech. While text-based approaches
for toxicity detection are common, there is limited research on processing
speech signals in the physical world. Detecting toxicity in the physical world
is challenging due to the difficulty of integrating AI-capable computers into
the environment. We propose a lightweight transformer model based on wav2vec2.0
and optimize it using techniques such as quantization and knowledge
distillation. Our model uses multitask learning and achieves an average macro
F1-score of 90.3\% and a weighted accuracy of 88\%, outperforming
state-of-the-art methods on DeToxy-B and a public dataset. Our results show
that quantization reduces the model size by almost 4 times and RAM usage by
3.3\%, with only a 1\% F1 score decrease. Knowledge distillation reduces the
model size by 3.7 times, RAM usage by 1.9, and inference time by 2 times, but
decreases accuracy by 8\%. Combining both techniques reduces the model size by
14.6 times and RAM usage by around 4.3 times, with a two-fold inference time
improvement. Our compact model is the first end-to-end speech-based toxicity
detection model based on a lightweight transformer model suitable for
deployment in physical spaces. The results show its feasibility for toxicity
detection on edge devices in real-world environments.
