  Gait recognition is a crucial biometric identification technique.
Camera-based gait recognition has been widely applied in both research and
industrial fields. LiDAR-based gait recognition has also begun to evolve most
recently, due to the provision of 3D structural information. However, in
certain applications, cameras fail to recognize persons, such as in low-light
environments and long-distance recognition scenarios, where LiDARs work well.
On the other hand, the deployment cost and complexity of LiDAR systems limit
its wider application. Therefore, it is essential to consider cross-modality
gait recognition between cameras and LiDARs for a broader range of
applications. In this work, we propose the first cross-modality gait
recognition framework between Camera and LiDAR, namely CL-Gait. It employs a
two-stream network for feature embedding of both modalities. This poses a
challenging recognition task due to the inherent matching between 3D and 2D
data, exhibiting significant modality discrepancy. To align the feature spaces
of the two modalities, i.e., camera silhouettes and LiDAR points, we propose a
contrastive pre-training strategy to mitigate modality discrepancy. To make up
for the absence of paired camera-LiDAR data for pre-training, we also introduce
a strategy for generating data on a large scale. This strategy utilizes
monocular depth estimated from single RGB images and virtual cameras to
generate pseudo point clouds for contrastive pre-training. Extensive
experiments show that the cross-modality gait recognition is very challenging
but still contains potential and feasibility with our proposed model and
pre-training strategy. To the best of our knowledge, this is the first work to
address cross-modality gait recognition.
