  Sampling-based methods, e.g., Deep Ensembles and Bayesian Neural Nets have
become promising approaches to improve the quality of uncertainty estimation
and robust generalization. However, they suffer from a large model size and
high latency at test-time, which limits the scalability needed for low-resource
devices and real-time applications. To resolve these computational issues, we
propose Density-Softmax, a sampling-free deterministic framework via combining
a density function built on a Lipschitz-constrained feature extractor with the
softmax layer. Theoretically, we show that our model is the solution of minimax
uncertainty risk and is distance-aware on feature space, thus reducing the
over-confidence of the standard softmax under distribution shifts. Empirically,
our method enjoys competitive results with state-of-the-art techniques in terms
of uncertainty and robustness, while having a lower number of model parameters
and a lower latency at test-time.
