  Very often features come with their own vectorial descriptions which provide
detailed information about their properties. We refer to these vectorial
descriptions as feature side-information. In the standard learning scenario,
input is represented as a vector of features and the feature side-information
is most often ignored or used only for feature selection prior to model
fitting. We believe that feature side-information which carries information
about features intrinsic property will help improve model prediction if used in
a proper way during learning process. In this paper, we propose a framework
that allows for the incorporation of the feature side-information during the
learning of very general model families to improve the prediction performance.
We control the structures of the learned models so that they reflect features
similarities as these are defined on the basis of the side-information. We
perform experiments on a number of benchmark datasets which show significant
predictive performance gains, over a number of baselines, as a result of the
exploitation of the side-information.
