  We define a novel neuro-symbolic framework, argumentative reward learning,
which combines preference-based argumentation with existing approaches to
reinforcement learning from human feedback. Our method improves prior work by
generalising human preferences, reducing the burden on the user and increasing
the robustness of the reward model. We demonstrate this with a number of
experiments.
