  We propose an interactive-predictive neural machine translation framework for
easier model personalization using reinforcement and imitation learning. During
the interactive translation process, the user is asked for feedback on
uncertain locations identified by the system. Responses are weak feedback in
the form of "keep" and "delete" edits, and expert demonstrations in the form of
"substitute" edits. Conditioning on the collected feedback, the system creates
alternative translations via constrained beam search. In simulation experiments
on two language pairs our systems get close to the performance of supervised
training with much less human effort.
