  Differential privacy (DP) defines privacy protection by promising quantified
indistinguishability between individuals that consent to share their
privacy-sensitive information and the ones that do not. DP aims to deliver this
promise by including well-crafted elements of random noise in the published
data and thus there is an inherent trade-off between the degree of privacy
protection and the ability to utilize the protected data. Currently, several
open-source tools were proposed for DP provision. To the best of our knowledge,
there is no comprehensive study for comparing these open-source tools with
respect to their ability to balance DP's inherent trade-off as well as the use
of system resources. This work proposes an open-source evaluation framework for
privacy protection solutions and offers evaluation for OpenDP Smartnoise,
Google DP, PyTorch Opacus, Tensorflow Privacy, and Diffprivlib. In addition to
studying their ability to balance the above trade-off, we consider discrete and
continuous attributes by quantifying their performance under different data
sizes. Our results reveal several patterns that developers should have in mind
when selecting tools under different application needs and criteria. This
evaluation survey can be the basis for an improved selection of open-source DP
tools and quicker adaptation of DP.
