  The main challenge in applying state-of-the-art deep learning methods to
predict image quality in-the-wild is the relatively small size of existing
quality scored datasets. The reason for the lack of larger datasets is the
massive resources required in generating diverse and publishable content. We
present a new systematic and scalable approach to create large-scale, authentic
and diverse image datasets for Image Quality Assessment (IQA). We show how we
built an IQA database, KonIQ-10k, consisting of 10,073 images, on which we
performed very large scale crowdsourcing experiments in order to obtain
reliable quality ratings from 1,467 crowd workers (1.2 million ratings). We
argue for its ecological validity by analyzing the diversity of the dataset, by
comparing it to state-of-the-art IQA databases, and by checking the reliability
of our user studies.
