  A solution manifold is the collection of points in a $d$-dimensional space
satisfying a system of $s$ equations with $s<d$. Solution manifolds occur in
several statistical problems including hypothesis testing, curved-exponential
families, constrained mixture models, partial identifications, and
nonparametric set estimation. We analyze solution manifolds both theoretically
and algorithmically. In terms of theory, we derive five useful results: the
smoothness theorem, the stability theorem (which implies the consistency of a
plug-in estimator), the convergence of a gradient flow, the local center
manifold theorem and the convergence of the gradient descent algorithm. To
numerically approximate a solution manifold, we propose a Monte Carlo gradient
descent algorithm. In the case of likelihood inference, we design a manifold
constraint maximization procedure to find the maximum likelihood estimator on
the manifold. We also develop a method to approximate a posterior distribution
defined on a solution manifold.
