  The deployment of Multi-Armed Bandits (MAB) has become commonplace in many
economic applications. However, regret guarantees for even state-of-the-art
linear bandit algorithms (such as Optimism in the Face of Uncertainty Linear
bandit (OFUL)) make strong exogeneity assumptions w.r.t. arm covariates. This
assumption is very often violated in many economic contexts and using such
algorithms can lead to sub-optimal decisions. Further, in social science
analysis, it is also important to understand the asymptotic distribution of
estimated parameters. To this end, in this paper, we consider the problem of
online learning in linear stochastic contextual bandit problems with endogenous
covariates. We propose an algorithm we term $\epsilon$-BanditIV, that uses
instrumental variables to correct for this bias, and prove an
$\tilde{\mathcal{O}}(k\sqrt{T})$ upper bound for the expected regret of the
algorithm. Further, we demonstrate the asymptotic consistency and normality of
the $\epsilon$-BanditIV estimator. We carry out extensive Monte Carlo
simulations to demonstrate the performance of our algorithms compared to other
methods. We show that $\epsilon$-BanditIV significantly outperforms other
existing methods in endogenous settings. Finally, we use data from real-time
bidding (RTB) system to demonstrate how $\epsilon$-BanditIV can be used to
estimate the causal impact of advertising in such settings and compare its
performance with other existing methods.
