  Graphics Processing Units (GPUs) are over-stressed to accelerate
High-Performance Computing applications and are used to accelerate Deep Neural
Networks in several domains where they have a life expectancy of many years.
These conditions expose the GPUs hardware to (premature) aging, causing
permanent faults to arise after the usual end-of-manufacturing test. Techniques
to assess the impact of permanent faults in GPUs are then strongly required,
thus allowing to estimate the reliability risk and to possibly mitigate it. In
this paper, we present a method to evaluate the effects of permanent faults
affecting the GPU scheduler and control units, which are the most peculiar and
stressed resources, along with the first figures that allow quantifying these
effects. We characterize over 5.83x10^5 permanent fault effects in the
scheduler and controllers of a gate-level GPU model. Then, we map the observed
error categories in software by instrumenting the code of 13 applications and
two convolutional neural networks, injecting more than 1.65x10^5 permanent
errors. Our two-level fault injection strategy reduces the evaluation time from
hundreds of years of gate-level evaluation to hundreds of hours.We found that
faults in the GPU parallelism management units can modify the opcode, the
addresses, and the status of thread(s) and warp(s). The large majority (up to
99%) of these hardware permanent errors impacts the running software execution.
Errors affecting the instruction operation or resource management hang the
code, while 45% of errors in the parallelism management or control-flow induce
silent data corruptions.
