  Interactive reinforcement learning has shown promise in learning complex
robotic tasks. However, the process can be human-intensive due to the
requirement of a large amount of interactive feedback. This paper presents a
new method that uses scores provided by humans instead of pairwise preferences
to improve the feedback efficiency of interactive reinforcement learning. Our
key insight is that scores can yield significantly more data than pairwise
preferences. Specifically, we require a teacher to interactively score the full
trajectories of an agent to train a behavioral policy in a sparse reward
environment. To avoid unstable scores given by humans negatively impacting the
training process, we propose an adaptive learning scheme. This enables the
learning paradigm to be insensitive to imperfect or unreliable scores. We
extensively evaluate our method for robotic locomotion and manipulation tasks.
The results show that the proposed method can efficiently learn near-optimal
policies by adaptive learning from scores while requiring less feedback
compared to pairwise preference learning methods. The source codes are publicly
available at https://github.com/SSKKai/Interactive-Scoring-IRL.
