  Recent advances in NLP show that language models retain a discernible level
of knowledge in deontological ethics and moral norms. However, existing works
often treat morality as binary, ranging from right to wrong. This simplistic
view does not capture the nuances of moral judgment. Pluralist moral
philosophers argue that human morality can be deconstructed into a finite
number of elements, respecting individual differences in moral judgment. In
line with this view, we build a pluralist moral sentence embedding space via a
state-of-the-art contrastive learning approach. We systematically investigate
the embedding space by studying the emergence of relationships among moral
elements, both quantitatively and qualitatively. Our results show that a
pluralist approach to morality can be captured in an embedding space. However,
moral pluralism is challenging to deduce via self-supervision alone and
requires a supervised approach with human labels.
