  Designing efficient general-purpose contextual bandit algorithms that work
with large -- or even continuous -- action spaces would facilitate application
to important scenarios such as information retrieval, recommendation systems,
and continuous control. While obtaining standard regret guarantees can be
hopeless, alternative regret notions have been proposed to tackle the large
action setting. We propose a smooth regret notion for contextual bandits, which
dominates previously proposed alternatives. We design a statistically and
computationally efficient algorithm -- for the proposed smooth regret -- that
works with general function approximation under standard supervised oracles. We
also present an adaptive algorithm that automatically adapts to any smoothness
level. Our algorithms can be used to recover the previous minimax/Pareto
optimal guarantees under the standard regret, e.g., in bandit problems with
multiple best arms and Lipschitz/H{\"o}lder bandits. We conduct large-scale
empirical evaluations demonstrating the efficacy of our proposed algorithms.
