  Given a set of hand-crafted local features, acquiring a global representation
via aggregation is a promising technique to boost computational efficiency and
improve task performance. Existing feature aggregation (FA) approaches,
including Bag of Words and Fisher Vectors, usually fail to capture the desired
information due to their pipeline mode. In this paper, we propose a generic
formulation to provide a systematical solution (named ProLFA) to aggregate
local descriptors. It is capable of producing compact yet interpretable
representations by selecting representative prototypes from numerous
descriptors, under relaxed exclusivity constraint. Meanwhile, to strengthen the
discriminability of the aggregated representation, we rationally enforce the
domain-invariant projection of bundled descriptors along a task-specific
direction. Furthermore, ProLFA is also provided with a powerful generalization
ability to deal flexibly with the semi-supervised and fully supervised
scenarios in local feature aggregation. Experimental results on various
descriptors and tasks demonstrate that the proposed ProLFA is considerably
superior over currently available alternatives about feature aggregation.
