  We consider numerical approaches for deterministic, finite-dimensional
optimal control problems whose dynamics depend on unknown or uncertain
parameters. We seek to amortize the solution over a set of relevant parameters
in an offline stage to enable rapid decision-making and be able to react to
changes in the parameter in the online stage. To tackle the curse of
dimensionality arising when the state and/or parameter are high-dimensional, we
represent the policy using neural networks. We compare two training paradigms:
First, our model-based approach leverages the dynamics and definition of the
objective function to learn the value function of the parameterized optimal
control problem and obtain the policy using a feedback form. Second, we use
actor-critic reinforcement learning to approximate the policy in a data-driven
way. Using an example involving a two-dimensional convection-diffusion
equation, which features high-dimensional state and parameter spaces, we
investigate the accuracy and efficiency of both training paradigms. While both
paradigms lead to a reasonable approximation of the policy, the model-based
approach is more accurate and considerably reduces the number of PDE solves.
