  The crux of learning vision-language models is to extract semantically
aligned information from visual and linguistic data. Existing attempts usually
face the problem of coarse alignment, e.g., the vision encoder struggles in
localizing an attribute-specified object. In this work, we propose an
embarrassingly simple approach to better align image and text features with no
need of additional data formats other than image-text pairs. Concretely, given
an image and its paired text, we manage to parse objects (e.g., cat) and
attributes (e.g., black) from the description, which are highly likely to exist
in the image. It is noteworthy that the parsing pipeline is fully automatic and
thus enjoys good scalability. With these parsed semantics as supervision
signals, we can complement the commonly used image-text contrastive loss with
the multi-tag classification loss. Extensive experimental results on a broad
suite of semantic segmentation datasets substantiate the average 5.2\%
improvement of our framework over existing alternatives. Furthermore, the
visualization results indicate that attribute supervision makes vision-language
models accurately localize attribute-specified objects. Project page can be
found at https://qinying-liu.github.io/Tag-Align.
