  This paper proposes an approach to content-preserving stitching of images
with regular boundary constraints, which aims to stitch multiple images to
generate a panoramic image with regular boundary. Existing methods treat image
stitching and rectangling as two separate steps, which may result in suboptimal
results as the stitching process is not aware of the further warping needs for
rectangling. We address these limitations by formulating image stitching with
regular boundaries in a unified optimization. Starting from the initial
stitching results produced by traditional warping-based optimization, we obtain
the irregular boundary from the warped meshes by polygon Boolean operations
which robustly handle arbitrary mesh compositions, and by analyzing the
irregular boundary construct a piecewise rectangular boundary. Based on this,
we further incorporate straight line preserving and regular boundary
constraints into the image stitching framework, and conduct iterative
optimization to obtain an optimal piecewise rectangular boundary, thus can make
the panoramic boundary as close as possible to a rectangle, while reducing
unwanted distortions. We further extend our method to panoramic videos and
selfie photography, by integrating the temporal coherence and portrait
preservation into the optimization. Experiments show that our method
efficiently produces visually pleasing panoramas with regular boundaries and
unnoticeable distortions.
