  This paper provides a novel parsimonious yet efficient design for zero-shot
learning (ZSL), dubbed ParsNets, where we are interested in learning a
composition of on-device friendly linear networks, each with orthogonality and
low-rankness properties, to achieve equivalent or even better performance
against existing deep models. Concretely, we first refactor the core module of
ZSL, i.e., visual-semantics mapping function, into several base linear networks
that correspond to diverse components of the semantic space, where the complex
nonlinearity can be collapsed into simple local linearities. Then, to
facilitate the generalization of local linearities, we construct a maximal
margin geometry on the learned features by enforcing low-rank constraints on
intra-class samples and high-rank constraints on inter-class samples, resulting
in orthogonal subspaces for different classes and each subspace lies on a
compact manifold. To enhance the model's adaptability and counterbalance
over/under-fittings in ZSL, a set of sample-wise indicators is employed to
select a sparse subset from these base linear networks to form a composite
semantic predictor for each sample. Notably, maximal margin geometry can
guarantee the diversity of features, and meanwhile, local linearities guarantee
efficiency. Thus, our ParsNets can generalize better to unseen classes and can
be deployed flexibly on resource-constrained devices. Theoretical explanations
and extensive experiments are conducted to verify the effectiveness of the
proposed method.
