  Conditional gradient algorithms (also often called Frank-Wolfe algorithms)
are popular due to their simplicity of only requiring a linear optimization
oracle and more recently they also gained significant traction for online
learning. While simple in principle, in many cases the actual implementation of
the linear optimization oracle is costly. We show a general method to lazify
various conditional gradient algorithms, which in actual computations leads to
several orders of magnitude of speedup in wall-clock time. This is achieved by
using a faster separation oracle instead of a linear optimization oracle,
relying only on few linear optimization oracle calls.
