  In recent centralized nonconvex distributed learning and federated learning,
local methods are one of the promising approaches to reduce communication time.
However, existing work has mainly focused on studying first-order optimality
guarantees. On the other side, second-order optimality guaranteed algorithms,
i.e., algorithms escaping saddle points, have been extensively studied in the
non-distributed optimization literature. In this paper, we study a new local
algorithm called Bias-Variance Reduced Local Perturbed SGD (BVR-L-PSGD), that
combines the existing bias-variance reduced gradient estimator with parameter
perturbation to find second-order optimal points in centralized nonconvex
distributed optimization. BVR-L-PSGD enjoys second-order optimality with nearly
the same communication complexity as the best known one of BVR-L-SGD to find
first-order optimality. Particularly, the communication complexity is better
than non-local methods when the local datasets heterogeneity is smaller than
the smoothness of the local loss. In an extreme case, the communication
complexity approaches to $\widetilde \Theta(1)$ when the local datasets
heterogeneity goes to zero. Numerical results validate our theoretical
findings.
