  Emotional voice conversion (EVC) seeks to convert the emotional state of an
utterance while preserving the linguistic content and speaker identity. In EVC,
emotions are usually treated as discrete categories overlooking the fact that
speech also conveys emotions with various intensity levels that the listener
can perceive. In this paper, we aim to explicitly characterize and control the
intensity of emotion. We propose to disentangle the speaker style from
linguistic content and encode the speaker style into a style embedding in a
continuous space that forms the prototype of emotion embedding. We further
learn the actual emotion encoder from an emotion-labelled database and study
the use of relative attributes to represent fine-grained emotion intensity. To
ensure emotional intelligibility, we incorporate emotion classification loss
and emotion embedding similarity loss into the training of the EVC network. As
desired, the proposed network controls the fine-grained emotion intensity in
the output speech. Through both objective and subjective evaluations, we
validate the effectiveness of the proposed network for emotional expressiveness
and emotion intensity control.
