  Automatic live video commenting is with increasing attention due to its
significance in narration generation, topic explanation, etc. However, the
diverse sentiment consideration of the generated comments is missing from the
current methods. Sentimental factors are critical in interactive commenting,
and lack of research so far. Thus, in this paper, we propose a
Sentiment-oriented Transformer-based Variational Autoencoder (So-TVAE) network
which consists of a sentiment-oriented diversity encoder module and a batch
attention module, to achieve diverse video commenting with multiple sentiments
and multiple semantics. Specifically, our sentiment-oriented diversity encoder
elegantly combines VAE and random mask mechanism to achieve semantic diversity
under sentiment guidance, which is then fused with cross-modal features to
generate live video comments. Furthermore, a batch attention module is also
proposed in this paper to alleviate the problem of missing sentimental samples,
caused by the data imbalance, which is common in live videos as the popularity
of videos varies. Extensive experiments on Livebot and VideoIC datasets
demonstrate that the proposed So-TVAE outperforms the state-of-the-art methods
in terms of the quality and diversity of generated comments. Related code is
available at https://github.com/fufy1024/So-TVAE.
