  A novel model called error loss network (ELN) is proposed to build an error
loss function for supervised learning. The ELN is in structure similar to a
radial basis function (RBF) neural network, but its input is an error sample
and output is a loss corresponding to that error sample. That means the
nonlinear input-output mapper of ELN creates an error loss function. The
proposed ELN provides a unified model for a large class of error loss
functions, which includes some information theoretic learning (ITL) loss
functions as special cases. The activation function, weight parameters and
network size of the ELN can be predetermined or learned from the error samples.
On this basis, we propose a new machine learning paradigm where the learning
process is divided into two stages: first, learning a loss function using an
ELN; second, using the learned loss function to continue to perform the
learning. Experimental results are presented to demonstrate the desirable
performance of the new method.
