  Gradient inversion attacks aim to reconstruct local training data from
intermediate gradients exposed in the federated learning framework. Despite
successful attacks, all previous methods, starting from reconstructing a single
data point and then relaxing the single-image limit to batch level, are only
tested under hard label constraints. Even for single-image reconstruction, we
still lack an analysis-based algorithm to recover augmented soft labels. In
this work, we change the focus from enlarging batchsize to investigating the
hard label constraints, considering a more realistic circumstance where label
smoothing and mixup techniques are used in the training process. In particular,
we are the first to initiate a novel algorithm to simultaneously recover the
ground-truth augmented label and the input feature of the last fully-connected
layer from single-input gradients, and provide a necessary condition for any
analytical-based label recovery methods. Extensive experiments testify to the
label recovery accuracy, as well as the benefits to the following image
reconstruction. We believe soft labels in classification tasks are worth
further attention in gradient inversion attacks.
