  We propose a new stochastic dual coordinate ascent technique that can be
applied to a wide range of regularized learning problems. Our method is based
on Alternating Direction Multiplier Method (ADMM) to deal with complex
regularization functions such as structured regularizations. Although the
original ADMM is a batch method, the proposed method offers a stochastic update
rule where each iteration requires only one or few sample observations.
Moreover, our method can naturally afford mini-batch update and it gives speed
up of convergence. We show that, under mild assumptions, our method converges
exponentially. The numerical experiments show that our method actually performs
efficiently.
