  Code comments are important for developers in program comprehension. In
scenarios of comprehending and reusing a method, developers expect code
comments to provide supplementary information beyond the method signature.
However, the extent of such supplementary information varies a lot in different
code comments. In this paper, we raise the awareness of the supplementary
nature of method-level comments and propose a new metric named MESIA (Mean
Supplementary Information Amount) to assess the extent of supplementary
information that a code comment can provide. With the MESIA metric, we conduct
experiments on a popular code-comment dataset and three common types of neural
approaches to generate method-level comments. Our experimental results
demonstrate the value of our proposed work with a number of findings. (1)
Small-MESIA comments occupy around 20% of the dataset and mostly fall into only
the WHAT comment category. (2) Being able to provide various kinds of essential
information, large-MESIA comments in the dataset are difficult for existing
neural approaches to generate. (3) We can improve the capability of existing
neural approaches to generate large-MESIA comments by reducing the proportion
of small-MESIA comments in the training set. (4) The retrained model can
generate large-MESIA comments that convey essential meaningful supplementary
information for methods in the small-MESIA test set, but will get a lower BLEU
score in evaluation. These findings indicate that with good training data,
auto-generated comments can sometimes even surpass human-written reference
comments, and having no appropriate ground truth for evaluation is an issue
that needs to be addressed by future work on automatic comment generation.
