  With the growing demand for synthetic data to address contemporary issues in
machine learning, such as data scarcity, data fairness, and data privacy,
having robust tools for assessing the utility and potential privacy risks of
such data becomes crucial. SynthEval, a novel open-source evaluation framework
distinguishes itself from existing tools by treating categorical and numerical
attributes with equal care, without assuming any special kind of preprocessing
steps. This~makes it applicable to virtually any synthetic dataset of tabular
records. Our tool leverages statistical and machine learning techniques to
comprehensively evaluate synthetic data fidelity and privacy-preserving
integrity. SynthEval integrates a wide selection of metrics that can be used
independently or in highly customisable benchmark configurations, and can
easily be extended with additional metrics. In this paper, we describe
SynthEval and illustrate its versatility with examples. The framework
facilitates better benchmarking and more consistent comparisons of model
capabilities.
