  Sliced-Wasserstein distance (SW) and its variant, Max Sliced-Wasserstein
distance (Max-SW), have been used widely in the recent years due to their fast
computation and scalability even when the probability measures lie in a very
high dimensional space. However, SW requires many unnecessary projection
samples to approximate its value while Max-SW only uses the most important
projection, which ignores the information of other useful directions. In order
to account for these weaknesses, we propose a novel distance, named
Distributional Sliced-Wasserstein distance (DSW), that finds an optimal
distribution over projections that can balance between exploring distinctive
projecting directions and the informativeness of projections themselves. We
show that the DSW is a generalization of Max-SW, and it can be computed
efficiently by searching for the optimal push-forward measure over a set of
probability measures over the unit sphere satisfying certain regularizing
constraints that favor distinct directions. Finally, we conduct extensive
experiments with large-scale datasets to demonstrate the favorable performances
of the proposed distances over the previous sliced-based distances in
generative modeling applications.
