  Continuous control is a widely applicable area of reinforcement learning. The
main players of this area are actor-critic methods that utilize policy
gradients of neural approximators as a common practice. The focus of our study
is to show the characteristics of the actor loss function which is the
essential part of the optimization. We exploit low dimensional visualizations
of the loss function and provide comparisons for loss landscapes of various
algorithms. Furthermore, we apply our approach to multi-store dynamic inventory
control, a notoriously difficult problem in supply chain operations, and
explore the shape of the loss function associated with the optimal policy. We
modelled and solved the problem using reinforcement learning while having a
loss landscape in favor of optimality.
