  Despite the decomposition of convolutional kernels for lightweight CNNs being
well studied, existing works that rely on tensor network diagrams or
hyperdimensional abstraction lack geometry intuition. This work devises a new
perspective by linking a 3D-reshaped kernel tensor to its various slice-wise
and rank-1 decompositions, permitting a straightforward connection between
various tensor approximations and efficient CNN modules. Specifically, it is
discovered that a pointwise-depthwise-pointwise (PDP) configuration constitutes
a viable construct for lightweight CNNs. Moreover, a novel link to the latest
ShiftNet is established, inspiring a first-ever shift layer pruning that
achieves nearly 50% compression with < 1% drop in accuracy for ShiftResNet.
