  Question Answering (QA) systems are increasingly deployed in applications
where they support real-world decisions. However, state-of-the-art models rely
on deep neural networks, which are difficult to interpret by humans. Inherently
interpretable models or post hoc explainability methods can help users to
comprehend how a model arrives at its prediction and, if successful, increase
their trust in the system. Furthermore, researchers can leverage these insights
to develop new methods that are more accurate and less biased. In this paper,
we introduce SQuARE v2, the new version of SQuARE, to provide an explainability
infrastructure for comparing models based on methods such as saliency maps and
graph-based explanations. While saliency maps are useful to inspect the
importance of each input token for the model's prediction, graph-based
explanations from external Knowledge Graphs enable the users to verify the
reasoning behind the model prediction. In addition, we provide multiple
adversarial attacks to compare the robustness of QA models. With these
explainability methods and adversarial attacks, we aim to ease the research on
trustworthy QA models. SQuARE is available on https://square.ukp-lab.de.
