  SemEval is the primary venue in the NLP community for the proposal of new
challenges and for the systematic empirical evaluation of NLP systems. This
paper provides a systematic quantitative analysis of SemEval aiming to evidence
the patterns of the contributions behind SemEval. By understanding the
distribution of task types, metrics, architectures, participation and citations
over time we aim to answer the question on what is being evaluated by SemEval.
