  Shared challenges provide a venue for comparing systems trained on common
data using a standardized evaluation, and they also provide an invaluable
resource for researchers when the data and evaluation results are publicly
released. The Blizzard Challenge and Voice Conversion Challenge are two such
challenges for text-to-speech synthesis and for speaker conversion,
respectively, and their publicly-available system samples and listening test
results comprise a historical record of state-of-the-art synthesis methods over
the years. In this paper, we revisit these past challenges and conduct a
large-scale listening test with samples from many challenges combined. Our aims
are to analyze and compare opinions of a large number of systems together, to
determine whether and how opinions change over time, and to collect a
large-scale dataset of a diverse variety of synthetic samples and their ratings
for further research. We found strong correlations challenge by challenge at
the system level between the original results and our new listening test. We
also observed the importance of the choice of speaker on synthesis quality.
