  We propose a system for tracking beats and downbeats with two objectives:
generality across a diverse music range, and high accuracy. We achieve
generality by training on multiple datasets -- including solo instrument
recordings, pieces with time signature changes, and classical music with high
tempo variations -- and by removing the commonly used Dynamic Bayesian Network
(DBN) postprocessing, which introduces constraints on the meter and tempo. For
high accuracy, among other improvements, we develop a loss function tolerant to
small time shifts of annotations, and an architecture alternating convolutions
with transformers either over frequency or time. Our system surpasses the
current state of the art in F1 score despite using no DBN. However, it can
still fail, especially for difficult and underrepresented genres, and performs
worse on continuity metrics, so we publish our model, code, and preprocessed
datasets, and invite others to beat this.
