  Transferability of learned features between tasks can massively reduce the
cost of training a neural network on a novel task. We investigate the effect of
network width on learned features using activation atlases --- a visualization
technique that captures features the entire hidden state responds to, as
opposed to individual neurons alone. We find that, while individual neurons do
not learn interpretable features in wide networks, groups of neurons do. In
addition, the hidden state of a wide network contains more information about
the inputs than that of a narrow network trained to the same test accuracy.
Inspired by this observation, we show that when fine-tuning the last layer of a
network on a new task, performance improves significantly as the width of the
network is increased, even though test accuracy on the original task is
independent of width.
