  We propose a novel end-to-end neural network architecture that, once trained,
directly outputs a probabilistic clustering of a batch of input examples in one
pass. It estimates a distribution over the number of clusters $k$, and for each
$1 \leq k \leq k_\mathrm{max}$, a distribution over the individual cluster
assignment for each data point. The network is trained in advance in a
supervised fashion on separate data to learn grouping by any perceptual
similarity criterion based on pairwise labels (same/different group). It can
then be applied to different data containing different groups. We demonstrate
promising performance on high-dimensional data like images (COIL-100) and
speech (TIMIT). We call this ``learning to cluster'' and show its conceptual
difference to deep metric learning, semi-supervise clustering and other related
approaches while having the advantage of performing learnable clustering fully
end-to-end.
