  Active learning can play an important role in low-resource settings (i.e.,
where annotated data is scarce), by selecting which instances may be more
worthy to annotate. Most active learning approaches for Machine Translation
assume the existence of a pool of sentences in a source language, and rely on
human annotators to provide translations or post-edits, which can still be
costly. In this article, we assume a real world human-in-the-loop scenario in
which: (i) the source sentences may not be readily available, but instead
arrive in a stream; (ii) the automatic translations receive feedback in the
form of a rating, instead of a correct/edited translation, since the
human-in-the-loop might be a user looking for a translation, but not be able to
provide one. To tackle the challenge of deciding whether each incoming pair
source-translations is worthy to query for human feedback, we resort to a
number of stream-based active learning query strategies. Moreover, since we not
know in advance which query strategy will be the most adequate for a certain
language pair and set of Machine Translation models, we propose to dynamically
combine multiple strategies using prediction with expert advice. Our
experiments show that using active learning allows to converge to the best
Machine Translation systems with fewer human interactions. Furthermore,
combining multiple strategies using prediction with expert advice often
outperforms several individual active learning strategies with even fewer
interactions.
