  Machine learning algorithms have difficulties to generalize over a small set
of examples. Humans can perform such a task by exploiting vast amount of
background knowledge they possess. One method for enhancing learning algorithms
with external knowledge is through feature generation. In this paper, we
introduce a new algorithm for generating features based on a collection of
auxiliary datasets. We assume that, in addition to the training set, we have
access to additional datasets. Unlike the transfer learning setup, we do not
assume that the auxiliary datasets represent learning tasks that are similar to
our original one. The algorithm finds features that are common to the training
set and the auxiliary datasets. Based on these features and examples from the
auxiliary datasets, it induces predictors for new features from the auxiliary
datasets. The induced predictors are then added to the original training set as
generated features. Our method was tested on a variety of learning tasks,
including text classification and medical prediction, and showed a significant
improvement over using just the given features.
