  Visual Attention Networks (VAN) with Large Kernel Attention (LKA) modules
have been shown to provide remarkable performance, that surpasses Vision
Transformers (ViTs), on a range of vision-based tasks. However, the depth-wise
convolutional layer in these LKA modules incurs a quadratic increase in the
computational and memory footprints with increasing convolutional kernel size.
To mitigate these problems and to enable the use of extremely large
convolutional kernels in the attention modules of VAN, we propose a family of
Large Separable Kernel Attention modules, termed LSKA. LSKA decomposes the 2D
convolutional kernel of the depth-wise convolutional layer into cascaded
horizontal and vertical 1-D kernels. In contrast to the standard LKA design,
the proposed decomposition enables the direct use of the depth-wise
convolutional layer with large kernels in the attention module, without
requiring any extra blocks. We demonstrate that the proposed LSKA module in VAN
can achieve comparable performance with the standard LKA module and incur lower
computational complexity and memory footprints. We also find that the proposed
LSKA design biases the VAN more toward the shape of the object than the texture
with increasing kernel size. Additionally, we benchmark the robustness of the
LKA and LSKA in VAN, ViTs, and the recent ConvNeXt on the five corrupted
versions of the ImageNet dataset that are largely unexplored in the previous
works. Our extensive experimental results show that the proposed LSKA module in
VAN provides a significant reduction in computational complexity and memory
footprints with increasing kernel size while outperforming ViTs, ConvNeXt, and
providing similar performance compared to the LKA module in VAN on object
recognition, object detection, semantic segmentation, and robustness tests.
