  Recent successes of Reinforcement Learning (RL) allow an agent to learn
policies that surpass human experts but suffers from being time-hungry and
data-hungry. By contrast, human learning is significantly faster because prior
and general knowledge and multiple information resources are utilized. In this
paper, we propose a Planner-Actor-Critic architecture for huMAN-centered
planning and learning (PACMAN), where an agent uses its prior, high-level,
deterministic symbolic knowledge to plan for goal-directed actions, and also
integrates the Actor-Critic algorithm of RL to fine-tune its behavior towards
both environmental rewards and human feedback. This work is the first unified
framework where knowledge-based planning, RL, and human teaching jointly
contribute to the policy learning of an agent. Our experiments demonstrate that
PACMAN leads to a significant jump-start at the early stage of learning,
converges rapidly and with small variance, and is robust to inconsistent,
infrequent, and misleading feedback.
