  Model selection in contextual bandits is an important complementary problem
to regret minimization with respect to a fixed model class. We consider the
simplest non-trivial instance of model-selection: distinguishing a simple
multi-armed bandit problem from a linear contextual bandit problem. Even in
this instance, current state-of-the-art methods explore in a suboptimal manner
and require strong "feature-diversity" conditions. In this paper, we introduce
new algorithms that a) explore in a data-adaptive manner, and b) provide model
selection guarantees of the form $\mathcal{O}(d^{\alpha} T^{1- \alpha})$ with
no feature diversity conditions whatsoever, where $d$ denotes the dimension of
the linear model and $T$ denotes the total number of rounds. The first
algorithm enjoys a "best-of-both-worlds" property, recovering two prior results
that hold under distinct distributional assumptions, simultaneously. The second
removes distributional assumptions altogether, expanding the scope for
tractable model selection. Our approach extends to model selection among nested
linear contextual bandits under some additional assumptions.
