  We focus on creating agents that act in alignment with socially beneficial
norms and values in interactive narratives or text-based games -- environments
wherein an agent perceives and interacts with a world through natural language.
Such interactive agents are often trained via reinforcement learning to
optimize task performance, even when such rewards may lead to agent behaviors
that violate societal norms -- causing harm either to the agent itself or other
entities in the environment. Social value alignment refers to creating agents
whose behaviors conform to expected moral and social norms for a given context
and group of people -- in our case, it means agents that behave in a manner
that is less harmful and more beneficial for themselves and others.
  We build on the Jiminy Cricket benchmark (Hendrycks et al. 2021), a set of 25
annotated interactive narratives containing thousands of morally salient
scenarios covering everything from theft and bodily harm to altruism. We
introduce the GALAD (Game-value ALignment through Action Distillation) agent
that uses the social commonsense knowledge present in specially trained
language models to contextually restrict its action space to only those actions
that are aligned with socially beneficial values. An experimental study shows
that the GALAD agent makes decisions efficiently enough to improve
state-of-the-art task performance by 4% while reducing the frequency of
socially harmful behaviors by 25% compared to strong contemporary value
alignment approaches.
