  Triangle counting is a fundamental building block in graph algorithms. In
this paper, we propose a block-based triangle counting algorithm to reduce data
movement during both sequential and parallel execution. Our block-based
formulation makes the algorithm naturally suitable for heterogeneous
architectures. The problem of partitioning the adjacency matrix of a graph is
well-studied. Our task decomposition goes one step further: it partitions the
set of triangles in the graph. By streaming these small tasks to compute
resources, we can solve problems that do not fit on a device. We demonstrate
the effectiveness of our approach by providing an implementation on a compute
node with multiple sockets, cores and GPUs. The current state-of-the-art in
triangle enumeration processes the Friendster graph in 2.1 seconds, not
including data copy time between CPU and GPU. Using that metric, our approach
is 20 percent faster. When copy times are included, our algorithm takes 3.2
seconds. This is 5.6 times faster than the fastest published CPU-only time.
