  Zero-shot learning (ZSL) is one of the most extreme forms of learning from
scarce labeled data. It enables predicting that images belong to classes for
which no labeled training instances are available. In this paper, we present a
new ZSL framework that leverages both label attribute side information and a
semantic label hierarchy. We present two methods, lifted zero-shot prediction
and a custom conditional random field (CRF) model, that integrate both forms of
side information. We propose benchmark tasks for this framework that focus on
making predictions across a range of semantic levels. We show that lifted
zero-shot prediction can dramatically outperform baseline methods when making
predictions within specified semantic levels, and that the probability
distribution provided by the CRF model can be leveraged to yield further
performance improvements when making unconstrained predictions over the
hierarchy.
