  Large language models (LLMs) has been effectively used for many computer
vision tasks, including image classification. In this paper, we present a
simple yet effective approach for zero-shot image classification using
multimodal LLMs. By employing multimodal LLMs, we generate comprehensive
textual representations from input images. These textual representations are
then utilized to generate fixed-dimensional features in a cross-modal embedding
space. Subsequently, these features are fused together to perform zero-shot
classification using a linear classifier. Our method does not require prompt
engineering for each dataset; instead, we use a single, straightforward, set of
prompts across all datasets. We evaluated our method on several datasets, and
our results demonstrate its remarkable effectiveness, surpassing benchmark
accuracy on multiple datasets. On average over ten benchmarks, our method
achieved an accuracy gain of 4.1 percentage points, with an increase of 6.8
percentage points on the ImageNet dataset, compared to prior methods. Our
findings highlight the potential of multimodal LLMs to enhance computer vision
tasks such as zero-shot image classification, offering a significant
improvement over traditional methods.
