  Traditional voice conversion(VC) has been focused on speaker identity
conversion for speech with a neutral expression. We note that emotional
expression plays an essential role in daily communication, and the emotional
style of speech can be speaker-dependent. In this paper, we study the technique
to jointly convert the speaker identity and speaker-dependent emotional style,
that is called expressive voice conversion. We propose a StarGAN-based
framework to learn a many-to-many mapping across different speakers, that takes
into account speaker-dependent emotional style without the need for parallel
data. To achieve this, we condition the generator on emotional style encoding
derived from a pre-trained speech emotion recognition(SER) model. The
experiments validate the effectiveness of our proposed framework in both
objective and subjective evaluations. To our best knowledge, this is the first
study on expressive voice conversion.
