  Topic-controllable summarization is an emerging research area with a wide
range of potential applications. However, existing approaches suffer from
significant limitations. For example, the majority of existing methods built
upon recurrent architectures, which can significantly limit their performance
compared to more recent Transformer-based architectures, while they also
require modifications to the model's architecture for controlling the topic. At
the same time, there is currently no established evaluation metric designed
specifically for topic-controllable summarization. This work proposes a new
topic-oriented evaluation measure to automatically evaluate the generated
summaries based on the topic affinity between the generated summary and the
desired topic. The reliability of the proposed measure is demonstrated through
appropriately designed human evaluation. In addition, we adapt topic embeddings
to work with powerful Transformer architectures and propose a novel and
efficient approach for guiding the summary generation through control tokens.
Experimental results reveal that control tokens can achieve better performance
compared to more complicated embedding-based approaches while also being
significantly faster.
