  Listener head generation centers on generating non-verbal behaviors (e.g.,
smile) of a listener in reference to the information delivered by a speaker. A
significant challenge when generating such responses is the non-deterministic
nature of fine-grained facial expressions during a conversation, which varies
depending on the emotions and attitudes of both the speaker and the listener.
To tackle this problem, we propose the Emotional Listener Portrait (ELP), which
treats each fine-grained facial motion as a composition of several discrete
motion-codewords and explicitly models the probability distribution of the
motions under different emotion in conversation. Benefiting from the
``explicit'' and ``discrete'' design, our ELP model can not only automatically
generate natural and diverse responses toward a given speaker via sampling from
the learned distribution but also generate controllable responses with a
predetermined attitude. Under several quantitative metrics, our ELP exhibits
significant improvements compared to previous methods.
