  StepMania is a popular open-source clone of a rhythm-based video game. As is
common in popular games, there is a large number of community-designed levels.
It is often difficult for players and level authors to determine the difficulty
level of such community contributions. In this work, we formalize and analyze
the difficulty prediction task on StepMania levels as an ordinal regression
(OR) task. We standardize a more extensive and diverse selection of this data
resulting in five data sets, two of which are extensions of previous work. We
evaluate many competitive OR and non-OR models, demonstrating that neural
network-based models significantly outperform the state of the art and that
StepMania-level data makes for an excellent test bed for deep OR models. We
conclude with a user experiment showing our trained models' superiority over
human labeling.
