  Min-max formulations have attracted great attention in the ML community due
to the rise of deep generative models and adversarial methods, while
understanding the dynamics of gradient algorithms for solving such formulations
has remained a grand challenge. As a first step, we restrict to bilinear
zero-sum games and give a systematic analysis of popular gradient updates, for
both simultaneous and alternating versions. We provide exact conditions for
their convergence and find the optimal parameter setup and convergence rates.
In particular, our results offer formal evidence that alternating updates
converge "better" than simultaneous ones.
