  We develop dynamical programming methods for the purpose of optimal control
of quantum states with convex constraints and concave cost and bequest
functions of the quantum state. We consider both open loop and feedback control
schemes, which correspond respectively to deterministic and stochastic Master
Equation dynamics. For the quantum feedback control scheme with continuous
non-demolition observations we exploit the separation theorem of filtering and
control aspects for quantum stochastic dynamics to derive a generalized
Hamilton-Jacobi-Bellman equation. If the control is restricted to only
Hamiltonian terms this is equivalent to a Hamilton-Jacobi equation with an
extra linear dissipative term. In this work, we consider, in particular, the
case when control is restricted to only observation. A controlled qubit is
considered as an example throughout the development of the formalism. Finally,
we discuss optimum observation strategies to obtain a pure state from a mixed
state of a quantum two-level system.
