  We present a novel approach for learning to predict sets using deep learning.
In recent years, deep neural networks have shown remarkable results in computer
vision, natural language processing and other related problems. Despite their
success, traditional architectures suffer from a serious limitation in that
they are built to deal with structured input and output data, i.e. vectors or
matrices. Many real-world problems, however, are naturally described as sets,
rather than vectors. Existing techniques that allow for sequential data, such
as recurrent neural networks, typically heavily depend on the input and output
order and do not guarantee a valid solution. Here, we derive in a principled
way, a mathematical formulation for set prediction where the output is
permutation invariant. In particular, our approach jointly learns both the
cardinality and the state distribution of the target set. We demonstrate the
validity of our method on the task of multi-label image classification and
achieve a new state of the art on the PASCAL VOC and MS COCO datasets.
