Reward Advancement: Transforming Policy under Maximum Causal Entropy
  Principle
  Many real-world human behaviors can be characterized as a sequential decision
making processes, such as urban travelers choices of transport modes and routes
(Wu et al. 2017). Differing from choices controlled by machines, which in
general follows perfect rationality to adopt the policy with the highest
reward, studies have revealed that human agents make sub-optimal decisions
under bounded rationality (Tao, Rohde, and Corcoran 2014). Such behaviors can
be modeled using maximum causal entropy (MCE) principle (Ziebart 2010). In this
paper, we define and investigate a general reward trans-formation problem
(namely, reward advancement): Recovering the range of additional reward
functions that transform the agent's policy from original policy to a
predefined target policy under MCE principle. We show that given an MDP and a
target policy, there are infinite many additional reward functions that can
achieve the desired policy transformation. Moreover, we propose an algorithm to
further extract the additional rewards with minimum "cost" to implement the
policy transformation.
