  We propose the differentially private lottery ticket mechanism (DPLTM). An
end-to-end differentially private training paradigm based on the lottery ticket
hypothesis. Using "high-quality winners", selected via our custom score
function, DPLTM significantly improves the privacy-utility trade-off over the
state-of-the-art. We show that DPLTM converges faster, allowing for early
stopping with reduced privacy budget consumption. We further show that the
tickets from DPLTM are transferable across datasets, domains, and
architectures. Our extensive evaluation on several public datasets provides
evidence to our claims.
