  The main goal of this paper is to apply the machinery of variational analysis
and generalized differentiation to study infinite horizon stochastic dynamic
programming (DP) with discrete time in the Banach space setting without
convexity assumptions. Unlike to standard stochastic DP with stationary Markov
processes, we investigate here stochastic DP in $L^p$ spaces to deal with
nonstationary stochastic processes, which describe a more flexible learning
procedure for the decision-maker. Our main concern is to calculate generalized
subgradients of the corresponding value function and to derive necessary
conditions for optimality in terms of the stochastic Euler inclusion under
appropriate Lipschitzian assumptions. The usage of the subdifferential formula
for integral functionals on $L^p$ spaces allows us, in particular, to find
verifiable conditions to ensure smoothness of the value function without any
convexity and/or interiority assumptions.
