  Let $\{U^N_t\}_{t\ge 0}$ be a standard Brownian motion on $\mathbb{U}(N)$.
For fixed $N\in\mathbb{N}$ and $t>0$, we give explicit bounds on the
$L_1$-Wasserstein distance of the empirical spectral measure of $U^N_t$ to both
the ensemble-averaged spectral measure and to the large-$N$ limiting measure
identified by Biane. We are then able to use these bounds to control the rate
of convergence of paths of the measures on compact time intervals. The proofs
use tools developed by the first author to study convergence rates of the
classical random matrix ensembles, as well as recent estimates for the
convergence of the moments of the ensemble-average spectral distribution.
