  We describe a general framework for online adaptation of optimization
hyperparameters by `hot swapping' their values during learning. We investigate
this approach in the context of adaptive learning rate selection using an
explore-exploit strategy from the multi-armed bandit literature. Experiments on
a benchmark neural network show that the hot swapping approach leads to
consistently better solutions compared to well-known alternatives such as
AdaDelta and stochastic gradient with exhaustive hyperparameter search.
