  Softmax is the de facto standard in modern neural networks for language
processing when it comes to normalizing logits. However, by producing a dense
probability distribution each token in the vocabulary has a nonzero chance of
being selected at each generation step, leading to a variety of reported
problems in text generation. $\alpha$-entmax of Peters et al. (2019,
arXiv:1905.05702) solves this problem, but is considerably slower than softmax.
  In this paper, we propose an alternative to $\alpha$-entmax, which keeps its
virtuous characteristics, but is as fast as optimized softmax and achieves on
par or better performance in machine translation task.
