  Multiclass multilabel classification is the task of attributing multiple
labels to examples via predictions. Current models formulate a reduction of the
multilabel setting into either multiple binary classifications or multiclass
classification, allowing for the use of existing loss functions (sigmoid,
cross-entropy, logistic, etc.). Multilabel classification reductions do not
accommodate for the prediction of varying numbers of labels per example and the
underlying losses are distant estimates of the performance metrics. We propose
a loss function, sigmoidF1, which is an approximation of the F1 score that (1)
is smooth and tractable for stochastic gradient descent, (2) naturally
approximates a multilabel metric, and (3) estimates label propensities and
label counts. We show that any confusion matrix metric can be formulated with a
smooth surrogate. We evaluate the proposed loss function on text and image
datasets, and with a variety of metrics, to account for the complexity of
multilabel classification evaluation. sigmoidF1 outperforms other loss
functions on one text and two image datasets and several metrics. These results
show the effectiveness of using inference-time metrics as loss functions for
non-trivial classification problems like multilabel classification.
