  Imagine an interesting multimodal interactive scenario that you can see,
hear, and chat with an AI-generated digital character, who is capable of
behaving like Sheldon from The Big Bang Theory, as a DEEP copy from appearance
to personality. Towards this fantastic multimodal chatting scenario, we propose
a novel task, named Deep Personalized Character Creation (DPCC): creating
multimodal chat personalized characters from multimodal data such as TV shows.
Specifically, given a single- or multi-modality input (text, audio, video), the
goal of DPCC is to generate a multi-modality (text, audio, video) response,
which should be well-matched the personality of a specific character such as
Sheldon, and of high quality as well. To support this novel task, we further
collect a character centric multimodal dialogue dataset, named Deep
Personalized Character Dataset (DPCD), from TV shows. DPCD contains
character-specific multimodal dialogue data of ~10k utterances and ~6 hours of
audio/video per character, which is around 10 times larger compared to existing
related datasets.On DPCD, we present a baseline method for the DPCC task and
create 5 Deep personalized digital Characters (DeepCharacters) from Big Bang TV
Shows. We conduct both subjective and objective experiments to evaluate the
multimodal response from DeepCharacters in terms of characterization and
quality. The results demonstrates that, on our collected DPCD dataset, the
proposed baseline can create personalized digital characters for generating
multimodal response.Our collected DPCD dataset, the code of data collection and
our baseline will be published soon.
