  Transformers have revolutionized deep learning and generative modeling,
enabling unprecedented advancements in natural language processing tasks.
However, the size of transformer models is increasing continuously, driven by
enhanced capabilities across various deep-learning tasks. This trend of
ever-increasing model size has given rise to new challenges in terms of memory
and computing requirements. Conventional computing platforms, including GPUs,
suffer from suboptimal performance due to the memory demands imposed by models
with millions/billions of parameters. The emerging chiplet-based platforms
provide a new avenue for compute- and data-intensive machine learning (ML)
applications enabled by a Network-on-Interposer (NoI). However, designing
suitable hardware accelerators for executing Transformer inference workloads is
challenging due to a wide variety of complex computing kernels in the
Transformer architecture. In this paper, we leverage chiplet-based
heterogeneous integration (HI) to design a high-performance and
energy-efficient multi-chiplet platform to accelerate transformer workloads. We
demonstrate that the proposed NoI architecture caters to the data access
patterns inherent in a transformer model. The optimized placement of the
chiplets and the associated NoI links and routers enable superior performance
compared to the state-of-the-art hardware accelerators. The proposed NoI-based
architecture demonstrates scalability across varying transformer models and
improves latency and energy efficiency by up to 22.8x and 5.36x respectively.
