  We propose a novel data-augmentation technique for neural machine translation
based on ROT-$k$ ciphertexts. ROT-$k$ is a simple letter substitution cipher
that replaces a letter in the plaintext with the $k$th letter after it in the
alphabet. We first generate multiple ROT-$k$ ciphertexts using different values
of $k$ for the plaintext which is the source side of the parallel data. We then
leverage this enciphered training data along with the original parallel data
via multi-source training to improve neural machine translation. Our method,
CipherDAug, uses a co-regularization-inspired training procedure, requires no
external data sources other than the original training data, and uses a
standard Transformer to outperform strong data augmentation techniques on
several datasets by a significant margin. This technique combines easily with
existing approaches to data augmentation, and yields particularly strong
results in low-resource settings.
