  In this paper, we consider the problem of dynamic programming when supremum
terms appear in the objective function. Such terms can represent overhead costs
associated with the underlying state variables. Specifically, this form of
optimization problem can be used to represent optimal scheduling of batteries
such as the Tesla Powerwall for electrical consumers subject to demand charges
- a charge based on the maximum rate of electricity consumption. These demand
charges reflect the cost to the utility of building and maintaining generating
capacity. Unfortunately, we show that dynamic programming problems with
supremum terms do not satisfy the principle of optimality. However, we also
show that the supremum is a special case of the class of forward separable
objective functions. To solve the dynamic programming problem, we propose a
general class of optimization problems with forward separable objectives. We
then show that for any problem in this class, there exists an augmented-state
dynamic programming problem which satisfies the principle of optimality and the
solutions to which yield solutions to the original forward separable problem.
We further generalize this approach to stochastic dynamic programming problems
and apply the results to the problem of optimal battery scheduling with demand
charges using a databased stochastic model for electricity usage and solar
generation by the consumer.
