  In this paper, we have worked on interpretability, trust, and understanding
of the decisions made by models in the form of classification tasks. The task
is divided into 3 subtasks. The first task consists of determining Binary
Sexism Detection. The second task describes the Category of Sexism. The third
task describes a more Fine-grained Category of Sexism. Our work explores
solving these tasks as a classification problem by fine-tuning
transformer-based architecture. We have performed several experiments with our
architecture, including combining multiple transformers, using domain adaptive
pretraining on the unlabelled dataset provided by Reddit and Gab, Joint
learning, and taking different layers of transformers as input to a
classification head. Our system (with team name Attention) was able to achieve
a macro F1 score of 0.839 for task A, 0.5835 macro F1 score for task B and
0.3356 macro F1 score for task C at the Codalab SemEval Competition. Later we
improved the accuracy of Task B to 0.6228 and Task C to 0.3693 in the test set.
