  CPU-based inference can be an alternative to off-chip accelerators, and
vector architectures are a promising option due to their efficiency. However,
the large design space of convolutional algorithms and hardware implementations
makes it challenging to select the best options. This paper presents ongoing
research into co-designing vector architectures for CPU-based CNN inference,
focusing on the im2col+GEMM and Winograd kernels. Using the Gem5 simulator, we
examine the impact of various hardware microarchitectural features on RISC-V
Vector and ARM-SVE ISAs. We also study the impact of several BLIS-like
algorithmic optimizations on im2col+GEMM. Our co-design study shows that longer
vector lengths and larger caches can improve performance by 5x with our
optimized CNN kernels, compared to a vector length of 512-bit and 1MB of L2
cache. For Winograd, we present a novel approach of inter-tile parallelization
that exploits longer vector lengths and offers high memory reuse, resulting in
up to 2.4x performance improvement for non-strided convolutional layers with
3x3 kernel size. Our study also shows that Winograd requires smaller cache
sizes compared to im2col+GEMM.
