  Document images are now widely captured by handheld devices such as mobile
phones. The OCR performance on these images are largely affected due to
geometric distortion of the document paper, diverse camera positions and
complex backgrounds. In this paper, we propose a simple yet effective approach
to rectify distorted document image by estimating control points and reference
points. After that, we use interpolation method between control points and
reference points to convert sparse mappings to backward mapping, and remap the
original distorted document image to the rectified image. Furthermore, control
points are controllable to facilitate interaction or subsequent adjustment. We
can flexibly select post-processing methods and the number of vertices
according to different application scenarios. Experiments show that our
approach can rectify document images with various distortion types, and yield
state-of-the-art performance on real-world dataset. This paper also provides a
training dataset based on control points for document dewarping. Both the code
and the dataset are released at
https://github.com/gwxie/Document-Dewarping-with-Control-Points.
