  Dual control explicitly addresses the problem of trading off active
exploration and exploitation in the optimal control of partially unknown
systems. While the problem can be cast in the framework of stochastic dynamic
programming, exact solutions are only tractable for discrete state and action
spaces of very small dimension due to a series of nested minimization and
expectation operations. We propose an approximate dual control method for
systems with continuous state and input domain based on a rollout dynamic
programming approach, splitting the control horizon into a dual and an
exploitation part. The dual part is approximated using a scenario tree
generated by sampling the process noise and the unknown system parameters, for
which the underlying distribution is updated via Bayesian estimation along the
horizon. In the exploitation part, we fix the resulting parameter estimate of
each scenario branch and compute an open-loop control sequence for the
remainder of the horizon. The key benefit of the proposed sampling-based
approximation is that it enables the formulation as one optimization problem
that computes a collection of control sequences over the scenario tree, leading
to a dual model predictive control formulation.
