  In binary classification problems, mainly two approaches have been proposed;
one is loss function approach and the other is uncertainty set approach. The
loss function approach is applied to major learning algorithms such as support
vector machine (SVM) and boosting methods. The loss function represents the
penalty of the decision function on the training samples. In the learning
algorithm, the empirical mean of the loss function is minimized to obtain the
classifier. Against a backdrop of the development of mathematical programming,
nowadays learning algorithms based on loss functions are widely applied to
real-world data analysis. In addition, statistical properties of such learning
algorithms are well-understood based on a lots of theoretical works. On the
other hand, the learning method using the so-called uncertainty set is used in
hard-margin SVM, mini-max probability machine (MPM) and maximum margin MPM. In
the learning algorithm, firstly, the uncertainty set is defined for each binary
label based on the training samples. Then, the best separating hyperplane
between the two uncertainty sets is employed as the decision function. This is
regarded as an extension of the maximum-margin approach. The uncertainty set
approach has been studied as an application of robust optimization in the field
of mathematical programming. The statistical properties of learning algorithms
with uncertainty sets have not been intensively studied. In this paper, we
consider the relation between the above two approaches. We point out that the
uncertainty set is described by using the level set of the conjugate of the
loss function. Based on such relation, we study statistical properties of
learning algorithms using uncertainty sets.
