  Humans subconsciously engage in geospatial reasoning when reading articles.
We recognize place names and their spatial relations in text and mentally
associate them with their physical locations on Earth. Although pretrained
language models can mimic this cognitive process using linguistic context, they
do not utilize valuable geospatial information in large, widely available
geographical databases, e.g., OpenStreetMap. This paper introduces GeoLM, a
geospatially grounded language model that enhances the understanding of
geo-entities in natural language. GeoLM leverages geo-entity mentions as
anchors to connect linguistic information in text corpora with geospatial
information extracted from geographical databases. GeoLM connects the two types
of context through contrastive learning and masked language modeling. It also
incorporates a spatial coordinate embedding mechanism to encode distance and
direction relations to capture geospatial context. In the experiment, we
demonstrate that GeoLM exhibits promising capabilities in supporting toponym
recognition, toponym linking, relation extraction, and geo-entity typing, which
bridge the gap between natural language processing and geospatial sciences. The
code is publicly available at https://github.com/knowledge-computing/geolm.
