  Privacy-preserving data release algorithms have gained increasing attention
for their ability to protect user privacy while enabling downstream machine
learning tasks. However, the utility of current popular algorithms is not
always satisfactory. Mixup of raw data provides a new way of data augmentation,
which can help improve utility. However, its performance drastically
deteriorates when differential privacy (DP) noise is added. To address this
issue, this paper draws inspiration from the recently observed Neural Collapse
(NC) phenomenon, which states that the last layer features of a neural network
concentrate on the vertices of a simplex as Equiangular Tight Frame (ETF). We
propose a scheme to mixup the Neural Collapse features to exploit the ETF
simplex structure and release noisy mixed features to enhance the utility of
the released data. By using Gaussian Differential Privacy (GDP), we obtain an
asymptotic rate for the optimal mixup degree. To further enhance the utility
and address the label collapse issue when the mixup degree is large, we propose
a Hierarchical sampling method to stratify the mixup samples on a small number
of classes. This method remarkably improves utility when the number of classes
is large. Extensive experiments demonstrate the effectiveness of our proposed
method in protecting against attacks and improving utility. In particular, our
approach shows significantly improved utility compared to directly training
classification networks with DPSGD on CIFAR100 and MiniImagenet datasets,
highlighting the benefits of using privacy-preserving data release. We release
reproducible code in https://github.com/Lidonghao1996/NeuroMixGDP.
