  We consider the problem of finding an $\varepsilon$-approximate stationary
point of a smooth function on a compact domain of $\mathbb{R}^d$. In contrast
with dimension-free approaches such as gradient descent, we focus here on the
case where $d$ is finite, and potentially small. This viewpoint was explored in
1993 by Vavasis, who proposed an algorithm which, for any fixed finite
dimension $d$, improves upon the $O(1/\varepsilon^2)$ oracle complexity of
gradient descent. For example for $d=2$, Vavasis' approach obtains the
complexity $O(1/\varepsilon)$. Moreover for $d=2$ he also proved a lower bound
of $\Omega(1/\sqrt{\varepsilon})$ for deterministic algorithms (we extend this
result to randomized algorithms).
  Our main contribution is an algorithm, which we call gradient flow trapping
(GFT), and the analysis of its oracle complexity. In dimension $d=2$, GFT
closes the gap with Vavasis' lower bound (up to a logarithmic factor), as we
show that it has complexity
$O\left(\sqrt{\frac{\log(1/\varepsilon)}{\varepsilon}}\right)$. In dimension
$d=3$, we show a complexity of
$O\left(\frac{\log(1/\varepsilon)}{\varepsilon}\right)$, improving upon
Vavasis' $O\left(1 / \varepsilon^{1.2} \right)$. In higher dimensions, GFT has
the remarkable property of being a logarithmic parallel depth strategy, in
stark contrast with the polynomial depth of gradient descent or Vavasis'
algorithm. In this higher dimensional regime, the total work of GFT improves
quadratically upon the only other known polylogarithmic depth strategy for this
problem, namely naive grid search. We augment this result with another
algorithm, named \emph{cut and flow} (CF), which improves upon Vavasis'
algorithm in any fixed dimension.
