  Bilevel optimization problems involve two nested objectives, where an
upper-level objective depends on a solution to a lower-level problem. When the
latter is non-convex, multiple critical points may be present, leading to an
ambiguous definition of the problem. In this paper, we introduce a key
ingredient for resolving this ambiguity through the concept of a selection map
which allows one to choose a particular solution to the lower-level problem.
Using such maps, we define a class of hierarchical games between two agents
that resolve the ambiguity in bilevel problems. This new class of games
requires introducing new analytical tools in Morse theory to extend implicit
differentiation, a technique used in bilevel optimization resulting from the
implicit function theorem. In particular, we establish the validity of such a
method even when the latter theorem is inapplicable due to degenerate critical
points. Finally, we show that algorithms for solving bilevel problems based on
unrolled optimization solve these games up to approximation errors due to
finite computational power. A simple correction to these algorithms is then
proposed for removing these errors.
