  One of the main challenges in Zero-Shot Learning of visual categories is
gathering semantic attributes to accompany images. Recent work has shown that
learning from textual descriptions, such as Wikipedia articles, avoids the
problem of having to explicitly define these attributes. We present a new model
that can classify unseen categories from their textual description.
Specifically, we use text features to predict the output weights of both the
convolutional and the fully connected layers in a deep convolutional neural
network (CNN). We take advantage of the architecture of CNNs and learn features
at different layers, rather than just learning an embedding space for both
modalities, as is common with existing approaches. The proposed model also
allows us to automatically generate a list of pseudo- attributes for each
visual category consisting of words from Wikipedia articles. We train our
models end-to-end us- ing the Caltech-UCSD bird and flower datasets and
evaluate both ROC and Precision-Recall curves. Our empirical results show that
the proposed model significantly outperforms previous methods.
