  This paper identifies and addresses dynamic selection problems in online
learning algorithms with endogenous data. In a contextual multi-armed bandit
model, a novel bias (self-fulfilling bias) arises because the endogeneity of
the data influences the choices of decisions, affecting the distribution of
future data to be collected and analyzed. We propose an
instrumental-variable-based algorithm to correct for the bias. It obtains true
parameter values and attains low (logarithmic-like) regret levels. We also
prove a central limit theorem for statistical inference. To establish the
theoretical properties, we develop a general technique that untangles the
interdependence between data and actions.
