  We study the continuity properties of optimal solutions to stochastic control
problems with respect to initial probability measures and applications of these
to the robustness of optimal control policies applied to systems with
incomplete or incorrect priors. It is shown that for single and multi-stage
optimal cost problems, continuity and robustness cannot be established under
weak convergence or Wasserstein convergence in general, but that the optimal
cost is continuous in the priors under the convergence in total variation under
mild conditions. By imposing further assumptions on the measurement models,
robustness and continuity also hold under weak convergence of priors. We thus
obtain robustness results and bounds on the mismatch error that occurs due to
the application of a control policy which is designed for an incorrectly
estimated prior in terms of a distance measure between the true prior and the
incorrect one. Positive and negative practical implications of these results in
empirical learning for stochastic control will be presented, where almost
surely weak convergence of i.i.d. empirical measures occurs but stronger
notions of convergence, such as total variation convergence, in general, do
not.
