  Convolution is one of the most computationally intensive operations that must
be performed for machine-learning model inference. A traditional approach to
compute convolutions is known as the Im2Col + BLAS method. This paper proposes
SConv: a direct-convolution algorithm based on a MLIR/LLVM code-generation
toolchain that can be integrated into machine-learning compilers . This
algorithm introduces: (a) Convolution Slicing Analysis (CSA) - a
convolution-specific 3D cache-blocking analysis pass that focuses on tile reuse
over the cache hierarchy; (b) Convolution Slicing Optimization (CSO) - a
code-generation pass that uses CSA to generate a tiled direct-convolution
macro-kernel; and (c) Vector-Based Packing (VBP) - an architecture-specific
optimized input-tensor packing solution based on vector-register shift
instructions for convolutions with unitary stride. Experiments conducted on 393
convolutions from full ONNX-MLIR machine-learning models indicate that the
elimination of the Im2Col transformation and the use of fast packing routines
result in a total packing time reduction, on full model inference, of 2.0x -
3.9x on Intel x86 and 3.6x - 7.2x on IBM POWER10. The speed-up over an Im2Col +
BLAS method based on current BLAS implementations for end-to-end
machine-learning model inference is in the range of 9% - 25% for Intel x86 and
10% - 42% for IBM POWER10 architectures. The total convolution speedup for
model inference is 12% - 27% on Intel x86 and 26% - 46% on IBM POWER10. SConv
also outperforms BLAS GEMM, when computing pointwise convolutions, in more than
83% of the 219 tested instances.
