  Social media platforms play an essential role in crisis communication, but
analyzing crisis-related social media texts is challenging due to their
informal nature. Transformer-based pre-trained models like BERT and RoBERTa
have shown success in various NLP tasks, but they are not tailored for
crisis-related texts. Furthermore, general-purpose sentence encoders are used
to generate sentence embeddings, regardless of the textual complexities in
crisis-related texts. Advances in applications like text classification,
semantic search, and clustering contribute to the effective processing of
crisis-related texts, which is essential for emergency responders to gain a
comprehensive view of a crisis event, whether historical or real-time. To
address these gaps in crisis informatics literature, this study introduces
CrisisTransformers, an ensemble of pre-trained language models and sentence
encoders trained on an extensive corpus of over 15 billion word tokens from
tweets associated with more than 30 crisis events, including disease outbreaks,
natural disasters, conflicts, and other critical incidents. We evaluate
existing models and CrisisTransformers on 18 crisis-specific public datasets.
Our pre-trained models outperform strong baselines across all datasets in
classification tasks, and our best-performing sentence encoder improves the
state-of-the-art by 17.43% in sentence encoding tasks. Additionally, we
investigate the impact of model initialization on convergence and evaluate the
significance of domain-specific models in generating semantically meaningful
sentence embeddings. The models are publicly available at:
https://huggingface.co/crisistransformers
