  We survey human evaluation in papers presenting work on creative natural
language generation that have been published in INLG 2020 and ICCC 2020. The
most typical human evaluation method is a scaled survey, typically on a 5 point
scale, while many other less common methods exist. The most commonly evaluated
parameters are meaning, syntactic correctness, novelty, relevance and emotional
value, among many others. Our guidelines for future evaluation include clearly
defining the goal of the generative system, asking questions as concrete as
possible, testing the evaluation setup, using multiple different evaluation
setups, reporting the entire evaluation process and potential biases clearly,
and finally analyzing the evaluation results in a more profound way than merely
reporting the most typical statistics.
