  Recent work [Ran22] formulated a class of optimal control problems involving
positive linear systems, linear stage costs, and elementwise constraints on
control. It was shown that the problem admits linear optimal cost and the
associated Bellman's equation can be characterized by a finite-dimensional
nonlinear equation, which is solved by linear programming. In this work, we
report exact dynamic programming (DP) theories for the same class of problems.
Moreover, we extend the results to a related class of problems where the norms
of control are bounded while the optimal costs remain linear. In both cases, we
provide conditions under which the solutions are unique, investigate properties
of the optimal policies, study the convergence of value iteration, policy
iteration, and optimistic policy iteration applied to such problems, and
analyze the boundedness of the solution to the associated optimization
programs. Apart from a form of the Frobenius-Perron theorem, the majority of
our results are built upon generic DP theory applicable to problems involving
nonnegative stage costs.
