<p>Maybe you can find a heuristic to detect when it gets stuck (e.g. gradient magnitude is too small, or it stays on the same triangle or returns to a recently visited triangle too many times in a row, or it hasn't moved at least X distance over the last N steps, etc) and just take a random step in some direction, e.g. to some randomly chosen neighboring triangle? That might help shake it out of the difficult region and get back to smooth descent.</p>
<p>For an even lower-tech solution, if you don't mind some randomness you could just inject occasional random moves all the time (without attempting to detect whether it's stuck). Kind of like stochastic gradient descent in ML.</p>
<p>If the gradient you're following is a geodesic distance, I think it can have saddle points, but not local minima (except for the global minimum), is that right? If that's a constraint you can rely on, you could try to use a second-order method, finding an estimate of the local <a href="https://en.wikipedia.org/wiki/Hessian_matrix" rel="nofollow noreferrer">Hessian</a> and looking for eigenvectors with negative eigenvalues, which should be <a href="https://en.wikipedia.org/wiki/Principal_curvature" rel="nofollow noreferrer">directions of principal curvature</a> that will accelerate the gradient descent, and in particular, will pull you out of saddle points. I'm sure there must be a literature on such methods although I don't know much about it.</p>

CC BY-SA 4.0
2020-09-09T20:32:57.383Z
I am just getting into the topic of computational geometry so excuse my ignorance. When you say looking for negative eigenvectors, I assume you mean negative eigenvectors of the hessian, which must mean the hessian is a matrix. Is that correct?
Makogan
CC BY-SA 4.0
2020-09-09T21:22:09.153Z
Yes (or in general, a linear operator). It contains all the second derivatives of the function along the mesh, just as the gradient contains the first derivatives. By finding the eigenvectors of the Hessian you get the principal curvature directions (of the graph of the function - not the curvatures of the mesh surface), and if one of those has negative curvature, that's the way out of the saddle point.
Nathan Reed
CC BY-SA 4.0
2020-09-09T18:28:09.170Z
2020-09-09T21:20:11.080Z
2020-09-09T21:20:11.080Z
Nathan Reed
<p>So I am doing a gradient descent like algorithm on the surface of a mesh and I just noticed something:</p>
<p><a href="https://i.sstatic.net/L3hx3.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/L3hx3.png" alt="enter image description here" /></a></p>
<p>The above is the geodesic gradient (the distance to a single vertex)</p>
<p>Look at where the ear connects with the head and notice how the discretized gradient behaves.
It's almost pointing in opposite directions. This is not however an error, this occurs because the gradient at that region is fairly &quot;turbulent&quot;.</p>
<p>My algorithm is fairly simple, grab a point in a triangle, compute its negative gradient, intersect the gradient with the triangle, move there, switch active triangle to the neighbour, repeat.</p>
<p>In other words you just follow the gradient down towards the source.</p>
<p>However, when I hit those turbulent regions, my algorithm gets stuck, because the gradient becomes essentially 0 and numerical errors prevent it from intersecting any of the edges of the triangle.</p>
<p>In pictures:</p>
<p><a href="https://i.sstatic.net/DgE3e.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/DgE3e.png" alt="enter image description here" /></a></p>
<p>The green points are the path taken by my gradient descent algorithm. The cyan ones are the centres of the faces of the triangles and the yellow arrow is the gradient at the last point.</p>
<p>Notice that this result is correct.</p>
<p>However, at the ears:</p>
<p><a href="https://i.sstatic.net/TVERW.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/TVERW.png" alt="enter image description here" /></a></p>
<p>As you can see the algorithm stops because the gradient is borderline 0.</p>
<p>And worse there are regions in the ears where the turbulence makes the gradient bounce and start travelling in the opposite direction.</p>
<p>Does anyone have tips on how I could handle the gradient in these regions?</p>

CC BY-SA 4.0
2020-09-10T03:33:59.503Z
You might wanna look into "Optimization on a manifold" which is literally the continuous analogous of what you're doing. Based on the manifold you work out "gradient","retraction" and "hessian", the use of the latter allows you to implement second order scheme which should behave better in theory. Discretizing these will give you the analogous on a mesh.
user8469759
<mathematics><computational-geometry><geometry><mesh><gradient>
Gradient descent (Not ML) on arbitrary meshes
Makogan
# Gradient descent (Not ML) on arbitrary meshes

(Asked by: username_0 on 2020-09-09)

So I am doing a gradient descent like algorithm on the surface of a mesh and I just noticed something:

[![enter image description here](https://i.sstatic.net/L3hx3.png)](https://i.sstatic.net/L3hx3.png)

The above is the geodesic gradient (the distance to a single vertex)

Look at where the ear connects with the head and notice how the discretized gradient behaves.
It's almost pointing in opposite directions. This is not however an error, this occurs because the gradient at that region is fairly "turbulent".

My algorithm is fairly simple, grab a point in a triangle, compute its negative gradient, intersect the gradient with the triangle, move there, switch active triangle to the neighbour, repeat.

In other words you just follow the gradient down towards the source.

However, when I hit those turbulent regions, my algorithm gets stuck, because the gradient becomes essentially 0 and numerical errors prevent it from intersecting any of the edges of the triangle.

In pictures:

[![enter image description here](https://i.sstatic.net/DgE3e.png)](https://i.sstatic.net/DgE3e.png)

The green points are the path taken by my gradient descent algorithm. The cyan ones are the centres of the faces of the triangles and the yellow arrow is the gradient at the last point.

Notice that this result is correct.

However, at the ears:

[![enter image description here](https://i.sstatic.net/TVERW.png)](https://i.sstatic.net/TVERW.png)

As you can see the algorithm stops because the gradient is borderline 0.

And worse there are regions in the ears where the turbulence makes the gradient bounce and start travelling in the opposite direction.

Does anyone have tips on how I could handle the gradient in these regions?

**Tags:** mathematics, computational-geometry, geometry, mesh, gradient

> **Comments:**
> **username_1**: You might wanna look into "Optimization on a manifold" which is literally the continuous analogous of what you're doing. Based on the manifold you work out "gradient","retraction" and "hessian", the use of the latter allows you to implement second order scheme which should behave better in theory. Discretizing these will give you the analogous on a mesh.  

---

## 1 answers

### Answer by username_2 on 2020-09-09. Score: 3 | Accepted answer

Maybe you can find a heuristic to detect when it gets stuck (e.g. gradient magnitude is too small, or it stays on the same triangle or returns to a recently visited triangle too many times in a row, or it hasn't moved at least X distance over the last N steps, etc) and just take a random step in some direction, e.g. to some randomly chosen neighboring triangle? That might help shake it out of the difficult region and get back to smooth descent.

For an even lower-tech solution, if you don't mind some randomness you could just inject occasional random moves all the time (without attempting to detect whether it's stuck). Kind of like stochastic gradient descent in ML.

If the gradient you're following is a geodesic distance, I think it can have saddle points, but not local minima (except for the global minimum), is that right? If that's a constraint you can rely on, you could try to use a second-order method, finding an estimate of the local [Hessian](https://en.wikipedia.org/wiki/Hessian_matrix) and looking for eigenvectors with negative eigenvalues, which should be [directions of principal curvature](https://en.wikipedia.org/wiki/Principal_curvature) that will accelerate the gradient descent, and in particular, will pull you out of saddle points. I'm sure there must be a literature on such methods although I don't know much about it.

> **Comments:**
> **username_0**: I am just getting into the topic of computational geometry so excuse my ignorance. When you say looking for negative eigenvectors, I assume you mean negative eigenvectors of the hessian, which must mean the hessian is a matrix. Is that correct?  
> **username_2**: Yes (or in general, a linear operator). It contains all the second derivatives of the function along the mesh, just as the gradient contains the first derivatives. By finding the eigenvectors of the Hessian you get the principal curvature directions (of the graph of the function - not the curvatures of the mesh surface), and if one of those has negative curvature, that's the way out of the saddle point.  

---

