  The past several years have witnessed the rapid progress of end-to-end Neural
Machine Translation (NMT). However, there exists discrepancy between training
and inference in NMT when decoding, which may lead to serious problems since
the model might be in a part of the state space it has never seen during
training. To address the issue, Scheduled Sampling has been proposed. However,
there are certain limitations in Scheduled Sampling and we propose two dynamic
oracle-based methods to improve it. We manage to mitigate the discrepancy by
changing the training process towards a less guided scheme and meanwhile
aggregating the oracle's demonstrations. Experimental results show that the
proposed approaches improve translation quality over standard NMT system.
