  A stagewise decomposition algorithm called value function gradient learning
(VFGL) is proposed for large-scale multistage stochastic convex programs. VFGL
finds the parameter values that best fit the gradient of the value function
within a given parametric family. Widely used decomposition algorithms for
multistage stochastic programming, such as stochastic dual dynamic programming
(SDDP), approximate the value function by adding linear subgradient cuts at
each iteration. Although this approach has been successful for linear problems,
nonlinear problems may suffer from the increasing size of each subproblem as
the iteration proceeds. On the other hand, VFGL has a fixed number of
parameters; thus, the size of the subproblems remains constant throughout the
iteration. Furthermore, VFGL can learn the parameters by means of stochastic
gradient descent, which means that it can be easily parallelized and does not
require a scenario tree approximation of the underlying uncertainties. VFGL was
compared with a deterministic equivalent formulation of the multistage
stochastic programming problem and SDDP approaches for three illustrative
examples: production planning, hydrothermal generation, and the lifetime
financial planning problem. Numerical examples show that VFGL generates
high-quality solutions and is computationally efficient.
