  Sparse matricized tensor times Khatri-Rao product (MTTKRP) is one of the most
computationally expensive kernels in sparse tensor computations. This work
focuses on optimizing the MTTKRP operation on GPUs, addressing both performance
and storage requirements. We begin by identifying the performance bottlenecks
in directly extending the state-of-the-art CSF (compressed sparse fiber) format
from CPUs to GPUs. A significant challenge with GPUs compared to multicore CPUs
is that of utilizing the much greater degree of parallelism in a load-balanced
fashion for irregular computations like sparse MTTKRP. To address this issue,
we develop a new storage-efficient representation for tensors that enables
high-performance, load-balanced execution of MTTKRP on GPUs. A GPU
implementation of sparse MTTKRP using the new sparse tensor representation is
shown to outperform all currently known parallel sparse CPU and GPU MTTKRP
implementations.
