  Evaluating the log-sum-exp function or the softmax function is a key step in
many modern data science algorithms, notably in inference and classification.
Because of the exponentials that these functions contain, the evaluation is
prone to overflow and underflow, especially in low precision arithmetic.
Software implementations commonly use alternative formulas that avoid overflow
and reduce the chance of harmful underflow, employing a shift or another
rewriting. Although mathematically equivalent, these variants behave
differently in floating-point arithmetic. We give rounding error analyses of
different evaluation algorithms and interpret the error bounds using condition
numbers for the functions. We conclude, based on the analysis and numerical
experiments, that the shifted formulas are of similar accuracy to the unshifted
ones and that the shifted softmax formula is typically more accurate than a
division-free variant.
