  For large-scale visual search, highly compressed yet meaningful
representations of images are essential. Structured vector quantizers based on
product quantization and its variants are usually employed to achieve such
compression while minimizing the loss of accuracy. Yet, unlike binary hashing
schemes, these unsupervised methods have not yet benefited from the
supervision, end-to-end learning and novel architectures ushered in by the deep
learning revolution. We hence propose herein a novel method to make deep
convolutional neural networks produce supervised, compact, structured binary
codes for visual search. Our method makes use of a novel block-softmax
non-linearity and of batch-based entropy losses that together induce structure
in the learned encodings. We show that our method outperforms state-of-the-art
compact representations based on deep hashing or structured quantization in
single and cross-domain category retrieval, instance retrieval and
classification. We make our code and models publicly available online.
