  Thanks to the power of representation learning, neural contextual bandit
algorithms demonstrate remarkable performance improvement against their
classical counterparts. But because their exploration has to be performed in
the entire neural network parameter space to obtain nearly optimal regret, the
resulting computational cost is prohibitively high. We perturb the rewards when
updating the neural network to eliminate the need of explicit exploration and
the corresponding computational overhead. We prove that a
$\tilde{O}(\tilde{d}\sqrt{T})$ regret upper bound is still achievable under
standard regularity conditions, where $T$ is the number of rounds of
interactions and $\tilde{d}$ is the effective dimension of a neural tangent
kernel matrix. Extensive comparisons with several benchmark contextual bandit
algorithms, including two recent neural contextual bandit models, demonstrate
the effectiveness and computational efficiency of our proposed neural bandit
algorithm.
