  This article reports on a survey carried out across the Natural Language
Processing (NLP) community. The survey aimed to capture the opinions of the
research community on issues surrounding shared tasks, with respect to both
participation and organisation. Amongst the 175 responses received, both
positive and negative observations were made. We carried out and report on an
extensive analysis of these responses, which leads us to propose a Shared Task
Organisation Checklist that could support future participants and organisers.
The proposed Checklist is flexible enough to accommodate the wide diversity of
shared tasks in our field and its goal is not to be prescriptive, but rather to
serve as a tool that encourages shared task organisers to foreground ethical
behaviour, beginning with the common issues that the 175 respondents deemed
important. Its usage would not only serve as an instrument to reflect on
important aspects of shared tasks, but would also promote increased
transparency around them.
