  This paper introduces a novel mathematical property applicable to diverse
images, referred to as FINOLA (First-Order Norm+Linear Autoregressive). FINOLA
represents each image in the latent space as a first-order autoregressive
process, in which each regression step simply applies a shared linear model on
the normalized value of its immediate neighbor. This intriguing property
reveals a mathematical invariance that transcends individual images. Expanding
from image grids to continuous coordinates, we unveil the presence of two
underlying partial differential equations. We validate the FINOLA property from
two distinct angles: image reconstruction and self-supervised learning.
Firstly, we demonstrate the ability of FINOLA to auto-regress up to a 256x256
feature map (the same resolution to the image) from a single vector placed at
the center, successfully reconstructing the original image by only using three
3x3 convolution layers as decoder. Secondly, we leverage FINOLA for
self-supervised learning by employing a simple masked prediction approach.
Encoding a single unmasked quadrant block, we autoregressively predict the
surrounding masked region. Remarkably, this pre-trained representation proves
highly effective in image classification and object detection tasks, even when
integrated into lightweight networks, all without the need for extensive
fine-tuning. The code will be made publicly available.
