  Transformer verification draws increasing attention in machine learning
research and industry. It formally verifies the robustness of transformers
against adversarial attacks such as exchanging words in a sentence with
synonyms. However, the performance of transformer verification is still not
satisfactory due to bound-centric computation which is significantly different
from standard neural networks. In this paper, we propose Faith, an efficient
framework for transformer verification on GPUs. We first propose a
semantic-aware computation graph transformation to identify semantic
information such as bound computation in transformer verification. We exploit
such semantic information to enable efficient kernel fusion at the computation
graph level. Second, we propose a verification-specialized kernel crafter to
efficiently map transformer verification to modern GPUs. This crafter exploits
a set of GPU hardware supports to accelerate verification specialized
operations which are usually memory-intensive. Third, we propose an
expert-guided autotuning to incorporate expert knowledge on GPU backends to
facilitate large search space exploration. Extensive evaluations show that
Faith achieves $2.1\times$ to $3.4\times$ ($2.6\times$ on average) speedup over
state-of-the-art frameworks.
