  In this paper we propose a hybrid architecture of actor-critic algorithms for
reinforcement learning in parameterized action space, which consists of
multiple parallel sub-actor networks to decompose the structured action space
into simpler action spaces along with a critic network to guide the training of
all sub-actor networks. While this paper is mainly focused on parameterized
action space, the proposed architecture, which we call hybrid actor-critic, can
be extended for more general action spaces which has a hierarchical structure.
We present an instance of the hybrid actor-critic architecture based on
proximal policy optimization (PPO), which we refer to as hybrid proximal policy
optimization (H-PPO). Our experiments test H-PPO on a collection of tasks with
parameterized action space, where H-PPO demonstrates superior performance over
previous methods of parameterized action reinforcement learning.
