  The linear programming (LP) approach is, together with value iteration and
policy iteration, one of the three fundamental methods to solve optimal control
problems in a dynamic programming setting. Despite its simple formulation,
versatility, and predisposition to be employed in model-free settings, the LP
approach has not enjoyed the same popularity as the other methods. The reason
is the often poor scalability of the exact LP approach and the difficulty to
obtain bounded solutions for a reasonable amount of constraints. We mitigate
these issues here, by investigating fundamental geometric features of the LP
and developing sufficient conditions to guarantee finite solutions with minimal
constraints. In the model-free context, we show that boundedness can be
guaranteed by a suitable choice of dataset and objective function.
