  We present a simple yet effective end-to-end trainable deep network with
geometry-inspired convolutional operators for detecting vanishing points in
images. Traditional convolutional neural networks rely on aggregating edge
features and do not have mechanisms to directly exploit the geometric
properties of vanishing points as the intersections of parallel lines. In this
work, we identify a canonical conic space in which the neural network can
effectively compute the global geometric information of vanishing points
locally, and we propose a novel operator named conic convolution that can be
implemented as regular convolutions in this space. This new operator explicitly
enforces feature extractions and aggregations along the structural lines and
yet has the same number of parameters as the regular 2D convolution. Our
extensive experiments on both synthetic and real-world datasets show that the
proposed operator significantly improves the performance of vanishing point
detection over traditional methods. The code and dataset have been made
publicly available at https://github.com/zhou13/neurvps.
