  When modeling geo-spatial data, it is critical to capture spatial
correlations for achieving high accuracy. Spatial Auto-Regression (SAR) is a
common tool used to model such data, where the spatial contiguity matrix (W)
encodes the spatial correlations. However, the efficacy of SAR is limited by
two factors. First, it depends on the choice of contiguity matrix, which is
typically not learnt from data, but instead, is assumed to be known apriori.
Second, it assumes that the observations can be explained by linear models. In
this paper, we propose a Convolutional Neural Network (CNN) framework to model
geo-spatial data (specifi- cally housing prices), to learn the spatial
correlations automatically. We show that neighborhood information embedded in
satellite imagery can be leveraged to achieve the desired spatial smoothing. An
additional upside of our framework is the relaxation of linear assumption on
the data. Specific challenges we tackle while implementing our framework
include, (i) how much of the neighborhood is relevant while estimating housing
prices? (ii) what is the right approach to capture multiple resolutions of
satellite imagery? and (iii) what other data-sources can help improve the
estimation of spatial correlations? We demonstrate a marked improvement of 57%
on top of the SAR baseline through the use of features from deep neural
networks for the cities of London, Birmingham and Liverpool.
