  We propose an implicit gradient based scheme for a constrained optimization
problem with nonconvex loss function, which can be potentially used to analyze
a variety of applications in machine learning, including meta-learning,
hyperparameter optimization, and reinforcement learning. The proposed algorithm
is based on the iterative differentiation (ITD) strategy. We extend the
convergence and rate analysis of the current literature of bilevel optimization
to a constrained bilevel structure with the motivation of learning under
constraints. For addressing bilevel optimization using any first-order scheme,
it involves the gradient of the inner-level optimal solution with respect to
the outer variable (implicit gradient). In this paper, taking into account of a
possible large-scale structure, we propose an efficient way of obtaining the
implicit gradient. We further provide error bounds with respect to the true
gradients. Further, we provide nonasymptotic rate results.
