  Bandit algorithms are widely used in sequential decision problems to maximize
the cumulative reward. One potential application is mobile health, where the
goal is to promote the user's health through personalized interventions based
on user specific information acquired through wearable devices. Important
considerations include the type of, and frequency with which data is collected
(e.g. GPS, or continuous monitoring), as such factors can severely impact app
performance and users' adherence. In order to balance the need to collect data
that is useful with the constraint of impacting app performance, one needs to
be able to assess the usefulness of variables. Bandit feedback data are
sequentially correlated, so traditional testing procedures developed for
independent data cannot apply. Recently, a statistical testing procedure was
developed for the actor-critic bandit algorithm. An actor-critic algorithm
maintains two separate models, one for the actor, the action selection policy,
and the other for the critic, the reward model. The performance of the
algorithm as well as the validity of the test are guaranteed only when the
critic model is correctly specified. However, misspecification is frequent in
practice due to incorrect functional form or missing covariates. In this work,
we propose a modified actor-critic algorithm which is robust to critic
misspecification and derive a novel testing procedure for the actor parameters
in this case.
