  In this study, we introduce a new problem raised by social media and
photojournalism, named Image Address Localization (IAL), which aims to predict
the readable textual address where an image was taken. Existing two-stage
approaches involve predicting geographical coordinates and converting them into
human-readable addresses, which can lead to ambiguity and be
resource-intensive. In contrast, we propose an end-to-end framework named
AddressCLIP to solve the problem with more semantics, consisting of two key
ingredients: i) image-text alignment to align images with addresses and scene
captions by contrastive learning, and ii) image-geography matching to constrain
image features with the spatial distance in terms of manifold learning.
Additionally, we have built three datasets from Pittsburgh and San Francisco on
different scales specifically for the IAL problem. Experiments demonstrate that
our approach achieves compelling performance on the proposed datasets and
outperforms representative transfer learning methods for vision-language
models. Furthermore, extensive ablations and visualizations exhibit the
effectiveness of the proposed method. The datasets and source code are
available at https://github.com/xsx1001/AddressCLIP.
