  A machine learning (ML) feature network is a graph that connects ML features
in learning tasks based on their similarity. This network representation allows
us to view feature vectors as functions on the network. By leveraging function
operations from Fourier analysis and from functional analysis, one can easily
generate new and novel features, making use of the graph structure imposed on
the feature vectors. Such network structures have previously been studied
implicitly in image processing and computational biology. We thus describe
feature networks as graph structures imposed on feature vectors, and provide
applications in machine learning. One application involves graph-based
generalizations of convolutional neural networks, involving structured deep
learning with hierarchical representations of features that have varying depth
or complexity. This extends also to learning algorithms that are able to
generate useful new multilevel features. Additionally, we discuss the use of
feature networks to engineer new features, which can enhance the expressiveness
of the model. We give a specific example of a deep tree-structured feature
network, where hierarchical connections are formed through feature clustering
and feed-forward learning. This results in low learning complexity and
computational efficiency. Unlike "standard" neural features which are limited
to modulated (thresholded) linear combinations of adjacent ones, feature
networks offer more general feedforward dependencies among features. For
example, radial basis functions or graph structure-based dependencies between
features can be utilized.
