  The Gumbel-Softmax probability distribution allows learning discrete tokens
in generative learning, while the Gumbel-Argmax probability distribution is
useful in learning discrete structures in discriminative learning. Despite the
efforts invested in optimizing these probability models, their statistical
properties are under-explored. In this work, we investigate their
representation properties and determine for which families of parameters these
probability distributions are complete, i.e., can represent any probability
distribution, and minimal, i.e., can represent a probability distribution
uniquely. We rely on convexity and differentiability to determine these
statistical conditions and extend this framework to general probability models,
such as Gaussian-Softmax and Gaussian-Argmax. We experimentally validate the
qualities of these extensions, which enjoy a faster convergence rate. We
conclude the analysis by identifying two sets of parameters that satisfy these
assumptions and thus admit a complete and minimal representation. Our
contribution is theoretical with supporting practical evaluation.
