  Deep predictive models of neuronal activity have recently enabled several new
discoveries about the selectivity and invariance of neurons in the visual
cortex. These models learn a shared set of nonlinear basis functions, which are
linearly combined via a learned weight vector to represent a neuron's function.
Such weight vectors, which can be thought as embeddings of neuronal function,
have been proposed to define functional cell types via unsupervised clustering.
However, as deep models are usually highly overparameterized, the learning
problem is unlikely to have a unique solution, which raises the question if
such embeddings can be used in a meaningful way for downstream analysis. In
this paper, we investigate how stable neuronal embeddings are with respect to
changes in model architecture and initialization. We find that $L_1$
regularization to be an important ingredient for structured embeddings and
develop an adaptive regularization that adjusts the strength of regularization
per neuron. This regularization improves both predictive performance and how
consistently neuronal embeddings cluster across model fits compared to uniform
regularization. To overcome overparametrization, we propose an iterative
feature pruning strategy which reduces the dimensionality of
performance-optimized models by half without loss of performance and improves
the consistency of neuronal embeddings with respect to clustering neurons. This
result suggests that to achieve an objective taxonomy of cell types or a
compact representation of the functional landscape, we need novel architectures
or learning techniques that improve identifiability. We will make our code
available at publication time.
