  Neural network approaches have recently shown to be effective in several
information retrieval (IR) tasks. However, neural approaches often require
large volumes of training data to perform effectively, which is not always
available. To mitigate the shortage of labeled data, training neural IR models
with weak supervision has been recently proposed and received considerable
attention in the literature. In weak supervision, an existing model
automatically generates labels for a large set of unlabeled data, and a machine
learning model is further trained on the generated "weak" data. Surprisingly,
it has been shown in prior art that the trained neural model can outperform the
weak labeler by a significant margin. Although these obtained improvements have
been intuitively justified in previous work, the literature still lacks
theoretical justification for the observed empirical findings. In this position
paper, we propose to theoretically study weak supervision, in particular for IR
tasks, e.g., learning to rank. We briefly review a set of our recent
theoretical findings that shed light on learning from weakly supervised data,
and provide guidelines on how train learning to rank models with weak
supervision.
