  Collaborative filtering (CF) has been proven to be one of the most effective
techniques for recommendation. Among all CF approaches, SimpleX is the
state-of-the-art method that adopts a novel loss function and a proper number
of negative samples. However, there is no work that optimizes SimpleX on
multi-core CPUs, leading to limited performance. To this end, we perform an
in-depth profiling and analysis of existing SimpleX implementations and
identify their performance bottlenecks including (1) irregular memory accesses,
(2) unnecessary memory copies, and (3) redundant computations. To address these
issues, we propose an efficient CF training system (called HEAT) that fully
enables the multi-level caching and multi-threading capabilities of modern
CPUs. Specifically, the optimization of HEAT is threefold: (1) It tiles the
embedding matrix to increase data locality and reduce cache misses (thus
reduces read latency); (2) It optimizes stochastic gradient descent (SGD) with
sampling by parallelizing vector products instead of matrix-matrix
multiplications, in particular the similarity computation therein, to avoid
memory copies for matrix data preparation; and (3) It aggressively reuses
intermediate results from the forward phase in the backward phase to alleviate
redundant computation. Evaluation on five widely used datasets with both x86-
and ARM-architecture processors shows that HEAT achieves up to 45.2X speedup
over existing CPU solution and 4.5X speedup and 7.9X cost reduction in Cloud
over existing GPU solution with NVIDIA V100 GPU.
