  Supervised, semi-supervised, and unsupervised learning estimate a function
given input/output samples. Generalization of the learned function to unseen
data can be improved by incorporating side information into learning. Side
information are data that are neither from the input space nor from the output
space of the function, but include useful information for learning it. In this
paper we show that learning with side information subsumes a variety of related
approaches, e.g. multi-task learning, multi-view learning and learning using
privileged information. Our main contributions are (i) a new perspective that
connects these previously isolated approaches, (ii) insights about how these
methods incorporate different types of prior knowledge, and hence implement
different patterns, (iii) facilitating the application of these methods in
novel tasks, as well as (iv) a systematic experimental evaluation of these
patterns in two supervised learning tasks.
