  Rigid image alignment is a fundamental task in computer vision, while the
traditional algorithms are either too sensitive to noise or time-consuming.
Recent unsupervised image alignment methods developed based on spatial
transformer networks show an improved performance on clean images but will not
achieve satisfactory performance on noisy images due to its heavy reliance on
pixel value comparations. To handle such challenging applications, we report a
new unsupervised difference learning (UDL) strategy and apply it to rigid image
alignment. UDL exploits the quantitative properties of regression tasks and
converts the original unsupervised problem to pseudo supervised problem. Under
the new UDL-based image alignment pipeline, rotation can be accurately
estimated on both clean and noisy images and translations can then be easily
solved. Experimental results on both nature and cryo-EM images demonstrate the
efficacy of our UDL-based unsupervised rigid image alignment method.
