  The wide-angle lens shows appealing applications in VR technologies, but it
introduces severe radial distortion into its captured image. To recover the
realistic scene, previous works devote to rectifying the content of the
wide-angle image. However, such a rectification solution inevitably distorts
the image boundary, which changes related geometric distributions and misleads
the current vision perception models. In this work, we explore constructing a
win-win representation on both content and boundary by contributing a new
learning model, i.e., Rectangling Rectification Network (RecRecNet). In
particular, we propose a thin-plate spline (TPS) module to formulate the
non-linear and non-rigid transformation for rectangling images. By learning the
control points on the rectified image, our model can flexibly warp the source
structure to the target domain and achieves an end-to-end unsupervised
deformation. To relieve the complexity of structure approximation, we then
inspire our RecRecNet to learn the gradual deformation rules with a DoF (Degree
of Freedom)-based curriculum learning. By increasing the DoF in each curriculum
stage, namely, from similarity transformation (4-DoF) to homography
transformation (8-DoF), the network is capable of investigating more detailed
deformations, offering fast convergence on the final rectangling task.
Experiments show the superiority of our solution over the compared methods on
both quantitative and qualitative evaluations. The code and dataset are
available at https://github.com/KangLiao929/RecRecNet.
