  Dual control denotes a class of control problems where the parameters
governing the system are imperfectly known. The challenge is to find the
optimal balance between probing, i.e. exciting the system to understand it
more, and caution, i.e. selecting conservative controls based on current
knowledge to achieve the control objective. Dynamic programming techniques can
achieve this optimal trade-off. However, while dynamic programming performs
well with discrete state and time, it is not well-suited to problems with
continuous time-frames or continuous or unbounded state spaces. Another
limitation is that multidimensional states often cause the dynamic programming
approaches to be intractable. In this paper, we investigate whether
continuous-time optimal control tools could help circumvent these caveats
whilst still achieving the probing-caution balance. We introduce a stylized
problem where the state is governed by one of two differential equations. It is
initially unknown which differential equation governs the system, so we must
simultaneously determine the true differential equation and control the system
to the desired state. We show how this problem can be transformed to apply
optimal control tools, and compare the performance of this approach to a
dynamic programming approach. Our results suggest that the optimal control
algorithm rivals dynamic programming on small problems, achieving the right
balance between aggressive and smoothly varying controls. In contrast to
dynamic programming, the optimal control approach remains tractable when
several states are to be controlled simultaneously.
