  Counseling is an activity of conversational speaking between a therapist and
a client. Therapist empathy is an essential indicator of counseling quality and
assessed subjectively by considering the entire conversation. This paper
proposes to encode long counseling conversation using a hierarchical attention
network. Conversations with extreme values of empathy rating are used to train
a Siamese network based encoder with contrastive loss. Two-level attention
mechanisms are applied to learn the importance weights of individual speaker
turns and groups of turns in the conversation. Experimental results show that
the use of contrastive loss is effective in encouraging the conversation
encoder to learn discriminative embeddings that are related to therapist
empathy. The distances between conversation embeddings positively correlate
with the differences in the respective empathy scores. The learned conversation
embeddings can be used to predict the subjective rating of therapist empathy.
