I know how to calculate PCA and SVD mathematically, and I know that both can be applied to Linear Least Squares regression.

The main advantage of SVD mathematically seems to be that it can be applied to non-square matrices.

Both focus on the decomposition of the $X^\top X$ matrix. Other than the advantage of SVD mentioned, are there any additional advantages or  insights provided by using SVD over PCA?

I'm really looking for the intuition rather than any mathematical differences.

As @ttnphns and @nick-cox said, SVD is a numerical method and PCA is an analysis approach (like least squares). You can do PCA using SVD, or you can do PCA doing the eigen-decomposition of $X^T X$ (or $X X^T$), or you can do PCA using many other methods, just like you can solve least squares with a dozen different algorithms like Newton's method or gradient descent or SVD etc.

So there is no "advantage" to SVD over PCA because it's like asking whether Newton's method is better than least squares: the two aren't comparable.
