  In recent years neural networks have achieved impressive results on many
technological and scientific tasks. Yet, the mechanism through which these
models automatically select features, or patterns in data, for prediction
remains unclear. Identifying such a mechanism is key to advancing performance
and interpretability of neural networks and promoting reliable adoption of
these models in scientific applications. In this paper, we identify and
characterize the mechanism through which deep fully connected neural networks
learn features. We posit the Deep Neural Feature Ansatz, which states that
neural feature learning occurs by implementing the average gradient outer
product to up-weight features strongly related to model output. Our ansatz
sheds light on various deep learning phenomena including emergence of spurious
features and simplicity biases and how pruning networks can increase
performance, the "lottery ticket hypothesis." Moreover, the mechanism
identified in our work leads to a backpropagation-free method for feature
learning with any machine learning model. To demonstrate the effectiveness of
this feature learning mechanism, we use it to enable feature learning in
classical, non-feature learning models known as kernel machines and show that
the resulting models, which we refer to as Recursive Feature Machines, achieve
state-of-the-art performance on tabular data.
