  Manipulating data, such as weighting data examples or augmenting with new
instances, has been increasingly used to improve model training. Previous work
has studied various rule- or learning-based approaches designed for specific
types of data manipulation. In this work, we propose a new method that supports
learning different manipulation schemes with the same gradient-based algorithm.
Our approach builds upon a recent connection of supervised learning and
reinforcement learning (RL), and adapts an off-the-shelf reward learning
algorithm from RL for joint data manipulation learning and model training.
Different parameterization of the "data reward" function instantiates different
manipulation schemes. We showcase data augmentation that learns a text
transformation network, and data weighting that dynamically adapts the data
sample importance. Experiments show the resulting algorithms significantly
improve the image and text classification performance in low data regime and
class-imbalance problems.
