  This paper describes a personality-adaptive multimodal dialogue system
developed for the Dialogue Robot Competition 2022. To realize a dialogue system
that adapts the dialogue strategy to individual users, it is necessary to
consider the user's nonverbal information and personality. In this competition,
we built a prototype of a user-adaptive dialogue system that estimates user
personality during dialogue. Pretrained DNN models are used to estimate user
personalities annotated as Big Five scores. This model is embedded in a
dialogue system to estimate user personality from face images during the
dialogue. We proposed a method for dialogue management that changed the
dialogue flow based on the estimated personality characteristics and confirmed
that the system works in a real environment in the preliminary round of this
competition. Furthermore, we implemented specific modules to enhance the
multimodal dialogue experience of the user, including personality assessment,
controlling facial expressions and movements of the android, and dialogue
management to explain the attractiveness of sightseeing spots. The aim of
dialogue based on personality assessment is to reduce the nervousness of users,
and it acts as an ice breaker. The android's facial expressions and movements
are necessary for a more natural android conversation. Since the task of this
competition was to promote the appeal of sightseeing spots and to recommend an
appropriate sightseeing spot, the dialogue process for how to explain the
attractiveness of the spot is important. All results of the subjective
evaluation by users were better than those of the baseline and other systems
developed for this competition. The proposed dialogue system ranked first in
both "Impression Rating" and "Effectiveness of Android Recommendations".
According to the total evaluation in the competition, the proposed system was
ranked first overall.
