  Scarcity of parallel sentence pairs is a major challenge for training high
quality neural machine translation (NMT) models in bilingually low-resource
scenarios, as NMT is data-hungry. Multi-task learning is an elegant approach to
inject linguistic-related inductive biases into NMT, using auxiliary syntactic
and semantic tasks, to improve generalisation. The challenge, however, is to
devise effective training schedules, prescribing when to make use of the
auxiliary tasks during the training process to fill the knowledge gaps of the
main translation task, a setting referred to as biased-MTL. Current approaches
for the training schedule are based on hand-engineering heuristics, whose
effectiveness vary in different MTL settings. We propose a novel framework for
learning the training schedule, ie learning to multi-task learn, for the MTL
setting of interest. We formulate the training schedule as a Markov decision
process which paves the way to employ policy learning methods to learn the
scheduling policy. We effectively and efficiently learn the training schedule
policy within the imitation learning framework using an oracle policy algorithm
that dynamically sets the importance weights of auxiliary tasks based on their
contributions to the generalisability of the main NMT task. Experiments on
low-resource NMT settings show the resulting automatically learned training
schedulers are competitive with the best heuristics, and lead to up to +1.1
BLEU score improvements.
