  In this paper we derive sufficient conditions for the convergence of two
popular alternating minimisation algorithms for dictionary learning - the
Method of Optimal Directions (MOD) and Online Dictionary Learning (ODL), which
can also be thought of as approximative K-SVD. We show that given a
well-behaved initialisation that is either within distance at most $1/\log(K)$
to the generating dictionary or has a special structure ensuring that each
element of the initialisation only points to one generating element, both
algorithms will converge with geometric convergence rate to the generating
dictionary. This is done even for data models with non-uniform distributions on
the supports of the sparse coefficients. These allow the appearance frequency
of the dictionary elements to vary heavily and thus model real data more
closely.
