  In this paper we provide an algorithmic framework based on Langevin diffusion
(LD) and its corresponding discretizations that allow us to simultaneously
obtain: i) An algorithm for sampling from the exponential mechanism, whose
privacy analysis does not depend on convexity and which can be stopped at
anytime without compromising privacy, and ii) tight uniform stability
guarantees for the exponential mechanism. As a direct consequence, we obtain
optimal excess empirical and population risk guarantees for (strongly) convex
losses under both pure and approximate differential privacy (DP). The framework
allows us to design a DP uniform sampler from the Rashomon set. Rashomon sets
are widely used in interpretable and robust machine learning, understanding
variable importance, and characterizing fairness.
