  CLIP has achieved impressive zero-shot performance after pre-training on a
large-scale dataset consisting of paired image-text data. Previous works have
utilized CLIP by incorporating manually designed visual prompts like colored
circles and blur masks into the images to guide the model's attention, showing
enhanced zero-shot performance in downstream tasks. Although these methods have
achieved promising results, they inevitably alter the original information of
the images, which can lead to failure in specific tasks. We propose a
train-free method Foveal-Attention CLIP (FALIP), which adjusts the CLIP's
attention by inserting foveal attention masks into the multi-head
self-attention module. We demonstrate FALIP effectively boosts CLIP zero-shot
performance in tasks such as referring expressions comprehension, image
classification, and 3D point cloud recognition. Experimental results further
show that FALIP outperforms existing methods on most metrics and can augment
current methods to enhance their performance.
