  We provide sharp path-dependent generalization and excess risk guarantees for
the full-batch Gradient Descent (GD) algorithm on smooth losses (possibly
non-Lipschitz, possibly nonconvex). At the heart of our analysis is an upper
bound on the generalization error, which implies that average output stability
and a bounded expected optimization error at termination lead to
generalization. This result shows that a small generalization error occurs
along the optimization path, and allows us to bypass Lipschitz or sub-Gaussian
assumptions on the loss prevalent in previous works. For nonconvex, convex, and
strongly convex losses, we show the explicit dependence of the generalization
error in terms of the accumulated path-dependent optimization error, terminal
optimization error, number of samples, and number of iterations. For nonconvex
smooth losses, we prove that full-batch GD efficiently generalizes close to any
stationary point at termination, and recovers the generalization error
guarantees of stochastic algorithms with fewer assumptions. For smooth convex
losses, we show that the generalization error is tighter than existing bounds
for SGD (up to one order of error magnitude). Consequently the excess risk
matches that of SGD for quadratically less iterations. Lastly, for strongly
convex smooth losses, we show that full-batch GD achieves essentially the same
excess risk rate as compared with the state of the art on SGD, but with an
exponentially smaller number of iterations (logarithmic in the dataset size).
