  Hate speech encompasses verbal, written, or behavioral communication that
targets derogatory or discriminatory language against individuals or groups
based on sensitive characteristics. Automated hate speech detection plays a
crucial role in curbing its propagation, especially across social media
platforms. Various methods, including recent advancements in deep learning,
have been devised to address this challenge. In this study, we introduce
HateTinyLLM, a novel framework based on fine-tuned decoder-only tiny large
language models (tinyLLMs) for efficient hate speech detection. Our
experimental findings demonstrate that the fine-tuned HateTinyLLM outperforms
the pretrained mixtral-7b model by a significant margin. We explored various
tiny LLMs, including PY007/TinyLlama-1.1B-step-50K-105b, Microsoft/phi-2, and
facebook/opt-1.3b, and fine-tuned them using LoRA and adapter methods. Our
observations indicate that all LoRA-based fine-tuned models achieved over 80\%
accuracy.
