  Common computational problems, such as parameter estimation in dynamic models
and PDE constrained optimization, require data fitting over a set of auxiliary
parameters subject to physical constraints over an underlying state. Naive
quadratically penalized formulations, commonly used in practice, suffer from
inherent ill-conditioning. We show that surprisingly the partial minimization
technique regularizes the problem, making it well-conditioned. This viewpoint
sheds new light on variable projection techniques, as well as the penalty
method for PDE constrained optimization, and motivates robust extensions. In
addition, we outline an inexact analysis, showing that the partial minimization
subproblem can be solved very loosely in each iteration. We illustrate the
theory and algorithms on boundary control, optimal transport, and parameter
estimation for robust dynamic inference.
