  Machine learning has an emerging critical role in high-performance computing
to modulate simulations, extract knowledge from massive data, and replace
numerical models with efficient approximations. Decision forests are a critical
tool because they provide insight into model operation that is critical to
interpreting learned results. While decision forests are trivially
parallelizable, the traversals of tree data structures incur many random memory
accesses and are very slow. We present memory packing techniques that
reorganize learned forests to minimize cache misses during classification. The
resulting layout is hierarchical. At low levels, we pack the nodes of multiple
trees into contiguous memory blocks so that each memory access fetches data for
multiple trees. At higher levels, we use leaf cardinality to identify the most
popular paths through a tree and collocate those paths in cache lines. We
extend this layout with out-of-order execution and cache-line prefetching to
increase memory throughput. Together, these optimizations increase the
performance of classification in ensembles by a factor of four over an
optimized C++ implementation and a actor of 50 over a popular R language
implementation.
