  The attention mechanism is a pivotal element within the Transformer
architecture, making a substantial contribution to its exceptional performance.
Within this attention mechanism, Softmax is an imperative component that
enables the model to assess the degree of correlation between various segments
of the input. Yet, prior research has shown that Softmax operations can
significantly increase processing latency and energy consumption in the
Transformer network due to their internal nonlinear operations and data
dependencies. In this work, we proposed~\textit{Hyft}, a hardware efficient
floating point Softmax accelerator for both training and inference. Hyft aims
to reduce the implementation cost of different nonlinear arithmetic operations
by adaptively converting intermediate results into the most suitable numeric
format for each specific operation, leading to reconfigurable accelerator with
hybrid numeric format. The evaluation results highlight that Hyft achieves a
remarkable $15\times$ reduction in hardware resource utilization and a $20
\times$ reduction in processing latency, all while maintaining a negligible
impact on Transformer accuracy.
