  This paper formalizes the binarization operations over neural networks from a
learning perspective. In contrast to classical hand crafted rules (\eg hard
thresholding) to binarize full-precision neurons, we propose to learn a mapping
from full-precision neurons to the target binary ones. Each individual weight
entry will not be binarized independently. Instead, they are taken as a whole
to accomplish the binarization, just as they work together in generating
convolution features. To help the training of the binarization mapping, the
full-precision neurons after taking sign operations is regarded as some
auxiliary supervision signal, which is noisy but still has valuable guidance.
An unbiased estimator is therefore introduced to mitigate the influence of the
supervision noise. Experimental results on benchmark datasets indicate that the
proposed binarization technique attains consistent improvements over baselines.
