  We study the problem of performing face verification with an efficient neural
model $f$. The efficiency of $f$ stems from simplifying the face verification
problem from an embedding nearest neighbor search into a binary problem; each
user has its own neural network $f$. To allow information sharing between
different individuals in the training set, we do not train $f$ directly but
instead generate the model weights using a hypernetwork $h$. This leads to the
generation of a compact personalized model for face identification that can be
deployed on edge devices. Key to the method's success is a novel way of
generating hard negatives and carefully scheduling the training objectives. Our
model leads to a substantially small $f$ requiring only 23k parameters and 5M
floating point operations (FLOPS). We use six face verification datasets to
demonstrate that our method is on par or better than state-of-the-art models,
with a significantly reduced number of parameters and computational burden.
Furthermore, we perform an extensive ablation study to demonstrate the
importance of each element in our method.
