  Deep feed-forward convolutional neural networks (CNNs) have become ubiquitous
in virtually all machine learning and computer vision challenges; however,
advancements in CNNs have arguably reached an engineering saturation point
where incremental novelty results in minor performance gains. Although there is
evidence that object classification has reached human levels on narrowly
defined tasks, for general applications, the biological visual system is far
superior to that of any computer. Research reveals there are numerous missing
components in feed-forward deep neural networks that are critical in mammalian
vision. The brain does not work solely in a feed-forward fashion, but rather
all of the neurons are in competition with each other; neurons are integrating
information in a bottom up and top down fashion and incorporating expectation
and feedback in the modeling process. Furthermore, our visual cortex is working
in tandem with our parietal lobe, integrating sensory information from various
modalities.
  In our work, we sought to improve upon the standard feed-forward deep
learning model by augmenting them with biologically inspired concepts of
sparsity, top-down feedback, and lateral inhibition. We define our model as a
sparse coding problem using hierarchical layers. We solve the sparse coding
problem with an additional top-down feedback error driving the dynamics of the
neural network. While building and observing the behavior of our model, we were
fascinated that multimodal, invariant neurons naturally emerged that mimicked,
"Halle Berry neurons" found in the human brain. Furthermore, our sparse
representation of multimodal signals demonstrates qualitative and quantitative
superiority to the standard feed-forward joint embedding in common vision and
machine learning tasks.
