  In text generation, a large language model (LM) makes a choice of each new
word based only on the former selection of its context using the softmax
function. Nevertheless, the link statistics information of concurrent words
based on a scene-specific corpus is valuable in choosing the next word, which
can help to ensure the topic of the generated text to be aligned with the
current task. To fully explore the co-occurrence information,we propose a
graphmax function for task-specific text generation. Using the graph-based
regularization, graphmax enables the final word choice to be determined by both
the global knowledge from the LM and the local knowledge from the
scene-specific corpus. The traditional softmax function is regularized with a
graph total variation (GTV) term, which incorporates the local knowledge into
the LM and encourages the model to consider the statistical relationships
between words in a scene-specific corpus. The proposed graphmax is versatile
and can be readily plugged into any large pre-trained LM for text generation
and machine translation. Through extensive experiments, we demonstrate that the
new GTV-based regularization can improve performances in various natural
language processing tasks in comparison with existing methods. Moreover,
through human experiments, we observe that participants can easily distinguish
the text generated by graphmax or softmax.
