  Deep neural networks are used for a wide range of regression problems.
However, there exists a significant gap in accuracy between specialized
approaches and generic direct regression in which a network is trained by
minimizing the squared or absolute error of output labels. Prior work has shown
that solving a regression problem with a set of binary classifiers can improve
accuracy by utilizing well-studied binary classification algorithms. We
introduce binary-encoded labels (BEL), which generalizes the application of
binary classification to regression by providing a framework for considering
arbitrary multi-bit values when encoding target values. We identify desirable
properties of suitable encoding and decoding functions used for the conversion
between real-valued and binary-encoded labels based on theoretical and
empirical study. These properties highlight a tradeoff between classification
error probability and error-correction capabilities of label encodings. BEL can
be combined with off-the-shelf task-specific feature extractors and trained
end-to-end. We propose a series of sample encoding, decoding, and training loss
functions for BEL and demonstrate they result in lower error than direct
regression and specialized approaches while being suitable for a diverse set of
regression problems, network architectures, and evaluation metrics. BEL
achieves state-of-the-art accuracies for several regression benchmarks. Code is
available at https://github.com/ubc-aamodt-group/BEL_regression.
