  Consider a convex function that is invariant under an group of
transformations. If it has a minimizer, does it also have an invariant
minimizer? Variants of this problem appear in nonparametric statistics and in a
number of adjacent fields. The answer depends on the choice of function, and on
what one may loosely call the geometry of the problem -- the interplay between
convexity, the group, and the underlying vector space, which is typically
infinite-dimensional. We observe that this geometry is completely encoded in
the smallest closed convex invariant subsets of the space, and proceed to study
these sets, for groups that are amenable but not necessarily compact. We then
apply this toolkit to the invariant optimality problem. It yields new results
on invariant kernel mean embeddings and risk-optimal invariant couplings, and
clarifies relations between seemingly distinct ideas, such as the summation
trick used in machine learning to construct equivariant neural networks and the
classic Hunt-Stein theorem of statistics.
