  In this paper, we propose a new approach called Adaptive Behavioral Costs in
Reinforcement Learning (ABC-RL) for training a human-like agent with
competitive strength. While deep reinforcement learning agents have recently
achieved superhuman performance in various video games, some of these
unconstrained agents may exhibit actions, such as shaking and spinning, that
are not typically observed in human behavior, resulting in peculiar gameplay
experiences. To behave like humans and retain similar performance, ABC-RL
augments behavioral limitations as cost signals in reinforcement learning with
dynamically adjusted weights. Unlike traditional constrained policy
optimization, we propose a new formulation that minimizes the behavioral costs
subject to a constraint of the value function. By leveraging the augmented
Lagrangian, our approach is an approximation of the Lagrangian adjustment,
which handles the trade-off between the performance and the human-like
behavior. Through experiments conducted on 3D games in DMLab-30 and Unity
ML-Agents Toolkit, we demonstrate that ABC-RL achieves the same performance
level while significantly reducing instances of shaking and spinning. These
findings underscore the effectiveness of our proposed approach in promoting
more natural and human-like behavior during gameplay.
