  Motivated by practical needs of experimentation and policy learning in online
platforms, we study the problem of safe data collection. Specifically, our goal
is to develop a logging policy that efficiently explores different actions to
elicit information while achieving competitive reward with a baseline
production policy. We first show that a common practice of mixing the
production policy with randomized exploration, despite being safe, is
sub-optimal in maximizing information gain. Then, we propose a safe optimal
logging policy via a novel water-filling technique for the case when no side
information about the actions' expected reward is available. We improve upon
this design by considering side information and also extend our approaches to
the linear contextual model to account for a large number of actions.
  Along the way, we analyze how our data logging policies impact errors in
off(line)-policy learning and empirically validate the benefit of our design by
conducting extensive numerical experiments with synthetic and MNIST datasets.
To further demonstrate the generality of our approach, we also consider the
safe online learning setting. By adaptively applying our techniques, we develop
the Safe Phased-Elimination (SafePE) algorithm that can achieve optimal regret
bound with only logarithmic number of policy updates.
