  We study policy optimization problems for deterministic Markov decision
processes (MDPs) with metric state and action spaces, which we refer to as
Metric Policy Optimization Problems (MPOPs). Our goal is to establish
theoretical results on the well-posedness of MPOPs that can characterize
practically relevant continuous control systems. To do so, we define a special
class of MPOPs called Compactly Restrictable MPOPs (CR-MPOPs), which are
flexible enough to capture the complex behavior of robotic systems but specific
enough to admit solutions using dynamic programming methods such as value
iteration. We show how to arrive at CR-MPOPs using forward-invariance. We
further show that our theoretical results on CR-MPOPs can be used to
characterize feedback linearizable control affine systems.
