  We propose a Safe Pontryagin Differentiable Programming (Safe PDP)
methodology, which establishes a theoretical and algorithmic framework to solve
a broad class of safety-critical learning and control tasks -- problems that
require the guarantee of safety constraint satisfaction at any stage of the
learning and control progress. In the spirit of interior-point methods, Safe
PDP handles different types of system constraints on states and inputs by
incorporating them into the cost or loss through barrier functions. We prove
three fundamentals of the proposed Safe PDP: first, both the solution and its
gradient in the backward pass can be approximated by solving their more
efficient unconstrained counterparts; second, the approximation for both the
solution and its gradient can be controlled for arbitrary accuracy by a barrier
parameter; and third, importantly, all intermediate results throughout the
approximation and optimization strictly respect the constraints, thus
guaranteeing safety throughout the entire learning and control process. We
demonstrate the capabilities of Safe PDP in solving various safety-critical
tasks, including safe policy optimization, safe motion planning, and learning
MPCs from demonstrations, on different challenging systems such as 6-DoF
maneuvering quadrotor and 6-DoF rocket powered landing.
