  Several applications in the scientific simulation of physical systems can be
formulated as control/optimization problems. The computational models for such
systems generally contain hyperparameters, which control solution fidelity and
computational expense. The tuning of these parameters is non-trivial and the
general approach is to manually `spot-check' for good combinations. This is
because optimal hyperparameter configuration search becomes impractical when
the parameter space is large and when they may vary dynamically. To address
this issue, we present a framework based on deep reinforcement learning (RL) to
train a deep neural network agent that controls a model solve by varying
parameters dynamically. First, we validate our RL framework for the problem of
controlling chaos in chaotic systems by dynamically changing the parameters of
the system. Subsequently, we illustrate the capabilities of our framework for
accelerating the convergence of a steady-state CFD solver by automatically
adjusting the relaxation factors of discretized Navier-Stokes equations during
run-time. The results indicate that the run-time control of the relaxation
factors by the learned policy leads to a significant reduction in the number of
iterations for convergence compared to the random selection of the relaxation
factors. Our results point to potential benefits for learning adaptive
hyperparameter learning strategies across different geometries and boundary
conditions with implications for reduced computational campaign expenses.
\footnote{Data and codes available at
\url{https://github.com/Romit-Maulik/PAR-RL}}
