  Set-based face recognition (SFR) aims to recognize the face sets in the
unconstrained scenario, where the appearance of same identity may change
dramatically with extreme variances (e.g., illumination, pose, expression). We
argue that the two crucial issues in SFR, the face quality and burstiness, are
both identity-irrelevant and variance-relevant. The quality and burstiness
assessment are interfered with by the entanglement of identity, and the face
recognition is interfered with by the entanglement of variance. Thus we propose
to separate the identity features with the variance features in a
light-weighted set-based disentanglement framework. Beyond disentanglement, the
variance features are fully utilized to indicate face quality and burstiness in
a set, rather than being discarded after training. To suppress face burstiness
in the sets, we propose a vocabulary-based burst suppression (VBS) method which
quantizes faces with a reference vocabulary. With interword and intra-word
normalization operations on the assignment scores, the face burtisness degrees
are appropriately estimated. The extensive illustrations and experiments
demonstrate the effect of the disentanglement framework with VBS, which gets
new state-of-the-art on the SFR benchmarks. The code will be released at
https://github.com/Liubinggunzu/set_burstiness.
