  In this paper, we consider a large-scale convex-concave saddle point problem
in a finite-sum form that arises in machine learning problems involving
empirical risk minimization, e.g., robust classification, kernel matrix
learning, etc. To contend with the challenges in computing full gradients, we
employ a block-coordinate primal-dual scheme in which randomly selected primal
and dual blocks of variables are updated at every iteration. When the number of
component functions in the finite sum is massive, we can derive further
efficiency by utilizing an increasing batch-size of gradients, leading to
progressive variance reduction. An ergodic convergence rate of
$\mathcal{O}(\log(K)/K)$ is obtained using variance reduction which is the
first known rate result for a special purpose method to deal with finite-sum
non-bilinear saddle point problems, matching the best-known rate (up to a log
factor) associated with deterministic primal-dual schemes using full gradients.
Using an acceleration for the setting where a single component gradient is
utilized, a non-asymptotic rate of $\mathcal{O}(\log(K)/\sqrt{K})$ is obtained.
Finally, for both single and increasing sample size scenarios, almost sure
convergence of the iterate sequence to a saddle point is shown.
