  The assumption that data samples are independently identically distributed is
the backbone of many learning algorithms. Nevertheless, datasets often exhibit
rich structure in practice, and we argue that there exist some unknown order
within the data instances. In this technical report, we introduce OrderNet that
can be used to extract the order of data instances in an unsupervised way. By
assuming that the instances are sampled from a Markov chain, our goal is to
learn the transitional operator of the underlying Markov chain, as well as the
order by maximizing the generation probability under all possible data
permutations. Specifically, we use neural network as a compact and soft lookup
table to approximate the possibly huge, but discrete transition matrix. This
strategy allows us to amortize the space complexity with a single model.
Furthermore, this simple and compact representation also provides a short
description to the dataset and generalizes to unseen instances as well. To
ensure that the learned Markov chain is ergodic, we propose a greedy batch-wise
permutation scheme that allows fast training. Empirically, we show that
OrderNet is able to discover an order among data instances. We also extend the
proposed OrderNet to one-shot recognition task and demonstrate favorable
results.
