  Automatic action quality assessment (AQA) has attracted increasing attention
due to its wide applications. However, most existing AQA methods employ
deterministic models to predict the final score for each action, while
overlooking the subjectivity and diversity among expert judges during the
scoring process. In this paper, we propose a novel probabilistic model, named
Uncertainty-Driven AQA (UD-AQA), to utilize and capture the diversity among
multiple judge scores. Specifically, we design a Conditional Variational
Auto-Encoder (CVAE)-based module to encode the uncertainty in expert
assessment, where multiple judge scores can be produced by sampling latent
features from the learned latent space multiple times. To further utilize the
uncertainty, we generate the estimation of uncertainty for each prediction,
which is employed to re-weight AQA regression loss, effectively reducing the
influence of uncertain samples during training. Moreover, we further design an
uncertainty-guided training strategy to dynamically adjust the learning order
of the samples from low uncertainty to high uncertainty. The experiments show
that our proposed method achieves competitive results on three benchmarks
including the Olympic events MTL-AQA and FineDiving, and the surgical skill
JIGSAWS datasets.
