  In this paper, we present a novel penalty model called ExPen for optimization
over the Stiefel manifold. Different from existing penalty functions for
orthogonality constraints, ExPen adopts a smooth penalty function without using
any first-order derivative of the objective function. We show that all the
first-order stationary points of ExPen with a sufficiently large penalty
parameter are either feasible, namely, are the first-order stationary points of
the original optimization problem, or far from the Stiefel manifold. Besides,
the original problem and ExPen share the same second-order stationary points.
Remarkably, the exact gradient and Hessian of ExPen are easy to compute. As a
consequence, abundant algorithm resources in unconstrained optimization can be
applied straightforwardly to solve ExPen.
