  Matrix factorization (MF) is employed by many popular algorithms, e.g.,
collaborative filtering. The emerging GPU technology, with massively multicore
and high intra-chip memory bandwidth but limited memory capacity, presents an
opportunity for accelerating MF much further when appropriately exploiting the
GPU architectural characteristics.
  This paper presents cuMF, a CUDA-based matrix factorization library that
implements memory-optimized alternate least square (ALS) method to solve very
large-scale MF. CuMF uses a variety set of techniques to maximize the
performance on either single or multiple GPUs. These techniques include smart
access of sparse data leveraging GPU memory hierarchy, using data parallelism
in conjunction with model parallelism, minimizing the communication overhead
between computing units, and utilizing a novel topology-aware parallel
reduction scheme.
  With only a single machine with four Nvidia GPU cards, cuMF can be 6-10 times
as fast, and 33-100 times as cost-efficient, compared with the state-of-art
distributed CPU solutions. Moreover, this cuMF can solve the largest matrix
factorization problem ever reported yet in current literature, while
maintaining impressively good performance.
