  Vision Transformers are being increasingly deployed in safety-critical
applications that demand high reliability. It is crucial to ensure the
correctness of their execution in spite of potential errors such as transient
hardware errors. We propose a novel algorithm-based resilience framework called
ALBERTA that allows us to perform end-to-end resilience analysis and protection
of transformer-based architectures. First, our work develops an efficient
process of computing and ranking the resilience of transformers layers. We find
that due to the large size of transformer models, applying traditional network
redundancy to a subset of the most vulnerable layers provides high error
coverage albeit with impractically high overhead. We address this shortcoming
by providing a software-directed, checksum-based error detection technique
aimed at protecting the most vulnerable general matrix multiply (GEMM) layers
in the transformer models that use either floating-point or integer arithmetic.
Results show that our approach achieves over 99% coverage for errors that
result in a mismatch with less than 0.2% and 0.01% computation and memory
overheads, respectively. Lastly, we present the applicability of our framework
in various modern GPU architectures under different numerical precisions. We
introduce an efficient self-correction mechanism for resolving erroneous
detection with an average of less than 2% overhead per error.
