  Scheduled sampling is widely used to mitigate the exposure bias problem for
neural machine translation. Its core motivation is to simulate the inference
scene during training by replacing ground-truth tokens with predicted tokens,
thus bridging the gap between training and inference. However, vanilla
scheduled sampling is merely based on training steps and equally treats all
decoding steps. Namely, it simulates an inference scene with uniform error
rates, which disobeys the real inference scene, where larger decoding steps
usually have higher error rates due to error accumulations. To alleviate the
above discrepancy, we propose scheduled sampling methods based on decoding
steps, increasing the selection chance of predicted tokens with the growth of
decoding steps. Consequently, we can more realistically simulate the inference
scene during training, thus better bridging the gap between training and
inference. Moreover, we investigate scheduled sampling based on both training
steps and decoding steps for further improvements. Experimentally, our
approaches significantly outperform the Transformer baseline and vanilla
scheduled sampling on three large-scale WMT tasks. Additionally, our approaches
also generalize well to the text summarization task on two popular benchmarks.
