  With a large amount of parallel data, neural machine translation systems are
able to deliver human-level performance for sentence-level translation.
However, it is costly to label a large amount of parallel data by humans. In
contrast, there is a large-scale of parallel corpus created by humans on the
Internet. The major difficulty to utilize them is how to filter them out from
the noise website environments. Current parallel data mining methods all
require labeled parallel data as the training source. In this paper, we present
a pipeline to mine the parallel corpus from the Internet in an unsupervised
manner. On the widely used WMT'14 English-French and WMT'16 English-German
benchmarks, the machine translator trained with the data extracted by our
pipeline achieves very close performance to the supervised results. On the
WMT'16 English-Romanian and Romanian-English benchmarks, our system produces
new state-of-the-art results, 39.81 and 38.95 BLEU scores, even compared with
supervised approaches.
