  Reliability of AI systems is a fundamental concern for the successful
deployment and widespread adoption of AI technologies. Unfortunately, the
escalating complexity and heterogeneity of AI hardware systems make them
increasingly susceptible to hardware faults, e.g., silent data corruptions
(SDC), that can potentially corrupt model parameters. When this occurs during
AI inference/servicing, it can potentially lead to incorrect or degraded model
output for users, ultimately affecting the quality and reliability of AI
services. In light of the escalating threat, it is crucial to address key
questions: How vulnerable are AI models to parameter corruptions, and how do
different components (such as modules, layers) of the models exhibit varying
vulnerabilities to parameter corruptions? To systematically address this
question, we propose a novel quantitative metric, Parameter Vulnerability
Factor (PVF), inspired by architectural vulnerability factor (AVF) in computer
architecture community, aiming to standardize the quantification of AI model
vulnerability against parameter corruptions. We define a model parameter's PVF
as the probability that a corruption in that particular model parameter will
result in an incorrect output. In this paper, we present several use cases on
applying PVF to three types of tasks/models during inference -- recommendation
(DLRM), vision classification (CNN), and text classification (BERT), while
presenting an in-depth vulnerability analysis on DLRM. PVF can provide pivotal
insights to AI hardware designers in balancing the tradeoff between fault
protection and performance/efficiency such as mapping vulnerable AI parameter
components to well-protected hardware modules. PVF metric is applicable to any
AI model and has a potential to help unify and standardize AI
vulnerability/resilience evaluation practice.
