  Assessing action quality is both imperative and challenging due to its
significant impact on the quality of AI-generated videos, further complicated
by the inherently ambiguous nature of actions within AI-generated video (AIGV).
Current action quality assessment (AQA) algorithms predominantly focus on
actions from real specific scenarios and are pre-trained with normative action
features, thus rendering them inapplicable in AIGVs. To address these problems,
we construct GAIA, a Generic AI-generated Action dataset, by conducting a
large-scale subjective evaluation from a novel causal reasoning-based
perspective, resulting in 971,244 ratings among 9,180 video-action pairs. Based
on GAIA, we evaluate a suite of popular text-to-video (T2V) models on their
ability to generate visually rational actions, revealing their pros and cons on
different categories of actions. We also extend GAIA as a testbed to benchmark
the AQA capacity of existing automatic evaluation methods. Results show that
traditional AQA methods, action-related metrics in recent T2V benchmarks, and
mainstream video quality methods correlate poorly with human opinions,
indicating a sizable gap between current models and human action perception
patterns in AIGVs. Our findings underscore the significance of action quality
as a unique perspective for studying AIGVs and can catalyze progress towards
methods with enhanced capacities for AQA in AIGVs.
