  We introduce several new black-box reductions that significantly improve the
design of adaptive and parameter-free online learning algorithms by simplifying
analysis, improving regret guarantees, and sometimes even improving runtime. We
reduce parameter-free online learning to online exp-concave optimization, we
reduce optimization in a Banach space to one-dimensional optimization, and we
reduce optimization over a constrained domain to unconstrained optimization.
All of our reductions run as fast as online gradient descent. We use our new
techniques to improve upon the previously best regret bounds for parameter-free
learning, and do so for arbitrary norms.
