  We develop a Monte-Carlo based numerical method for solving discrete-time
stochastic optimal control problems with inventory. These are optimal control
problems in which the control affects only a deterministically evolving
inventory process on a compact state space while the random underlying process
manifests itself through the objective functional. We propose a Regress Later
modification of the traditional Regression Monte Carlo which allows to decouple
inventory levels in two successive time steps and to include in the basis
functions of the regression the dependence on the inventory levels. We develop
a backward construction of trajectories for the inventory which enables us to
use policy iteration of Longstaff-Schwartz type avoiding nested simulations.
Our algorithm improves on the grid discretisation procedure largely used in
literature and practice, and on the recently proposed control randomisation by
[Kharroubi et al. (2014) Monte Carlo Methods and Applications, 20(2), pp.
145-165]. We validate our approach on three numerical examples: a benchmark
problem of energy arbitrage used to compare different methods available in
literature; a multi-dimensional problem of control of two connected water
reservoirs; and a high-dimensional problem of the management of a battery with
the purpose of assisting the operations of a wind turbine in providing
electricity to a group of buildings in a cost effective way.
