  Existing pedestrian attribute recognition (PAR) algorithms are mainly
developed based on a static image. However, the performance is not reliable for
images with challenging factors, such as heavy occlusion, motion blur, etc. In
this work, we propose to understand human attributes using video frames that
can make full use of temporal information. Specifically, we formulate the
video-based PAR as a vision-language fusion problem and adopt pre-trained big
models CLIP to extract the feature embeddings of given video frames. To better
utilize the semantic information, we take the attribute list as another input
and transform the attribute words/phrase into the corresponding sentence via
split, expand, and prompt. Then, the text encoder of CLIP is utilized for
language embedding. The averaged visual tokens and text tokens are concatenated
and fed into a fusion Transformer for multi-modal interactive learning. The
enhanced tokens will be fed into a classification head for pedestrian attribute
prediction. Extensive experiments on a large-scale video-based PAR dataset
fully validated the effectiveness of our proposed framework.
