  Recognizing and disentangling visual attributes from objects is a foundation
to many computer vision applications. While large vision language
representations like CLIP had largely resolved the task of zero-shot object
recognition, zero-shot visual attribute recognition remains a challenge because
CLIP's contrastively-learned vision-language representation cannot effectively
capture object-attribute dependencies. In this paper, we target this weakness
and propose a sentence generation-based retrieval formulation for attribute
recognition that is novel in 1) explicitly modeling a to-be-measured and
retrieved object-attribute relation as a conditional probability graph, which
converts the recognition problem into a dependency-sensitive language-modeling
problem, and 2) applying a large pretrained Vision-Language Model (VLM) on this
reformulation and naturally distilling its knowledge of image-object-attribute
relations to use towards attribute recognition. Specifically, for each
attribute to be recognized on an image, we measure the visual-conditioned
probability of generating a short sentence encoding the attribute's relation to
objects on the image. Unlike contrastive retrieval, which measures likelihood
by globally aligning elements of the sentence to the image, generative
retrieval is sensitive to the order and dependency of objects and attributes in
the sentence. We demonstrate through experiments that generative retrieval
consistently outperforms contrastive retrieval on two visual reasoning
datasets, Visual Attribute in the Wild (VAW), and our newly-proposed Visual
Genome Attribute Ranking (VGARank).
