  Model selection in the context of bandit optimization is a challenging
problem, as it requires balancing exploration and exploitation not only for
action selection, but also for model selection. One natural approach is to rely
on online learning algorithms that treat different models as experts. Existing
methods, however, scale poorly ($\text{poly}M$) with the number of models $M$
in terms of their regret. Our key insight is that, for model selection in
linear bandits, we can emulate full-information feedback to the online learner
with a favorable bias-variance trade-off. This allows us to develop ALEXP,
which has an exponentially improved ($\log M$) dependence on $M$ for its
regret. ALEXP has anytime guarantees on its regret, and neither requires
knowledge of the horizon $n$, nor relies on an initial purely exploratory
stage. Our approach utilizes a novel time-uniform analysis of the Lasso,
establishing a new connection between online learning and high-dimensional
statistics.
