  In this study, we propose a novel approach for investigating optimization
performance by flexible robot coordination in automated warehouses with
multi-agent reinforcement learning (MARL)-based control. Automated systems
using robots are expected to achieve efficient operations compared with manual
systems in terms of overall optimization performance. However, the impact of
overall optimization on performance remains unclear in most automated systems
due to a lack of suitable control methods. Thus, we proposed a centralized
training-and-decentralized execution MARL framework as a practical overall
optimization control method. In the proposed framework, we also proposed a
single shared critic, trained with global states and rewards, applicable to a
case in which heterogeneous agents make decisions asynchronously. Our proposed
MARL framework was applied to the task selection of material handling equipment
through automated order picking simulation, and its performance was evaluated
to determine how far overall optimization outperforms partial optimization by
comparing it with other MARL frameworks and rule-based control methods.
