  Several recent works in online optimization and game dynamics have
established strong negative complexity results including the formal emergence
of instability and chaos even in small such settings, e.g., $2\times 2$ games.
These results motivate the following question: Which methodological tools can
guarantee the regularity of such dynamics and how can we apply them in standard
settings of interest such as discrete-time first-order optimization dynamics?
We show how proving the existence of invariant functions, i.e., constant of
motions, is a fundamental contribution in this direction and establish a
plethora of such positive results (e.g. gradient descent, multiplicative
weights update, alternating gradient descent and manifold gradient descent)
both in optimization as well as in game settings. At a technical level, for
some conservation laws we provide an explicit and concise closed form, whereas
for other ones we present non-constructive proofs using tools from dynamical
systems.
