  Having sufficient resources for language X lifts it from the under-resourced
languages class, but not necessarily from the under-researched class. In this
paper, we address the problem of the absence of organized benchmarks in the
Turkish language. We demonstrate that languages such as Turkish are left behind
the state-of-the-art in NLP applications. As a solution, we present Mukayese, a
set of NLP benchmarks for the Turkish language that contains several NLP tasks.
We work on one or more datasets for each benchmark and present two or more
baselines. Moreover, we present four new benchmarking datasets in Turkish for
language modeling, sentence segmentation, and spell checking. All datasets and
baselines are available under: https://github.com/alisafaya/mukayese
