  This paper addresses the problem of document image dewarping, which aims at
eliminating the geometric distortion in document images for document
digitization. Instead of designing a better neural network to approximate the
optical flow fields between the inputs and outputs, we pursue the best
readability by taking the text lines and the document boundaries into account
from a constrained optimization perspective. Specifically, our proposed method
first learns the boundary points and the pixels in the text lines and then
follows the most simple observation that the boundaries and text lines in both
horizontal and vertical directions should be kept after dewarping to introduce
a novel grid regularization scheme. To obtain the final forward mapping for
dewarping, we solve an optimization problem with our proposed grid
regularization. The experiments comprehensively demonstrate that our proposed
approach outperforms the prior arts by large margins in terms of readability
(with the metrics of Character Errors Rate and the Edit Distance) while
maintaining the best image quality on the publicly-available DocUNet benchmark.
