  Transformers have captured growing attention in computer vision, thanks to
its large capacity and global processing capabilities. However, transformers
are data hungry, and their ability to generalize is constrained compared to
Convolutional Neural Networks (ConvNets), especially when trained with limited
data due to the absence of the built-in spatial inductive biases present in
ConvNets. In this paper, we strive to optimally combine the strengths of both
convolution and transformers for image classification tasks. Towards this end,
we present a novel lightweight hybrid network that pairs Convolution with
Transformers via Representation Learning Fusion and Multi-Level Feature
Cross-Attention named CTRL-F. Our network comprises a convolution branch and a
novel transformer module named multi-level feature cross-attention (MFCA). The
MFCA module operates on multi-level feature representations obtained at
different convolution stages. It processes small patch tokens and large patch
tokens extracted from these multi-level feature representations via two
separate transformer branches, where both branches communicate and exchange
knowledge through cross-attention mechanism. We fuse the local responses
acquired from the convolution path with the global responses acquired from the
MFCA module using novel representation fusion techniques dubbed adaptive
knowledge fusion (AKF) and collaborative knowledge fusion (CKF). Experiments
demonstrate that our CTRL-F variants achieve state-of-the-art performance,
whether trained from scratch on large data or even with low-data regime. For
Instance, CTRL-F achieves top-1 accuracy of 82.24% and 99.91% when trained from
scratch on Oxford-102 Flowers and PlantVillage datasets respectively,
surpassing state-of-the-art models which showcase the robustness of our model
on image classification tasks. Code at: https://github.com/hosamsherif/CTRL-F
