  This paper explores various attack scenarios on a voice anonymization system
using embeddings alignment techniques. We use Wasserstein-Procrustes (an
algorithm initially designed for unsupervised translation) or Procrustes
analysis to match two sets of x-vectors, before and after voice anonymization,
to mimic this transformation as a rotation function. We compute the optimal
rotation and compare the results of this approximation to the official Voice
Privacy Challenge results. We show that a complex system like the baseline of
the Voice Privacy Challenge can be approximated by a rotation, estimated using
a limited set of x-vectors. This paper studies the space of solutions for voice
anonymization within the specific scope of rotations. Rotations being
reversible, the proposed method can recover up to 62% of the speaker identities
from anonymized embeddings.
