  Estimating action quality, the process of assigning a "score" to the
execution of an action, is crucial in areas such as sports and health care.
Unlike action recognition, which has millions of examples to learn from, the
action quality datasets that are currently available are small -- typically
comprised of only a few hundred samples. This work presents three frameworks
for evaluating Olympic sports which utilize spatiotemporal features learned
using 3D convolutional neural networks (C3D) and perform score regression with
i) SVR, ii) LSTM, and iii) LSTM followed by SVR. An efficient training
mechanism for the limited data scenarios is presented for clip-based training
with LSTM. The proposed systems show significant improvement over existing
quality assessment approaches on the task of predicting scores of Olympic
events {diving, vault, figure skating}. While the SVR-based frameworks yield
better results, LSTM-based frameworks are more natural for describing an action
and can be used for improvement feedback.
