  We consider a discrete time stochastic Markovian control problem under model
uncertainty. Such uncertainty not only comes from the fact that the true
probability law of the underlying stochastic process is unknown, but the
parametric family of probability distributions which the true law belongs to is
also unknown. We propose a nonparametric adaptive robust control methodology to
deal with such problem. Our approach hinges on the following building concepts:
first, using the adaptive robust paradigm to incorporate online learning and
uncertainty reduction into the robust control problem; second, learning the
unknown probability law through the empirical distribution, and representing
uncertainty reduction in terms of a sequence of Wasserstein balls around the
empirical distribution; third, using Lagrangian duality to convert the
optimization over Wasserstein balls to a scalar optimization problem, and
adopting a machine learning technique to achieve efficient computation of the
optimal control. We illustrate our methodology by considering a utility
maximization problem. Numerical comparisons show that the nonparametric
adaptive robust control approach is preferable to the traditional robust
frameworks.
