  Even after decades of research, dynamic scene background reconstruction and
foreground object segmentation are still considered as open problems due
various challenges such as illumination changes, camera movements, or
background noise caused by air turbulence or moving trees. We propose in this
paper to model the background of a frame sequence as a low dimensional manifold
using an autoencoder and compare the reconstructed background provided by this
autoencoder with the original image to compute the foreground/background
segmentation masks. The main novelty of the proposed model is that the
autoencoder is also trained to predict the background noise, which allows to
compute for each frame a pixel-dependent threshold to perform the foreground
segmentation. Although the proposed model does not use any temporal or motion
information, it exceeds the state of the art for unsupervised background
subtraction on the CDnet 2014 and LASIESTA datasets, with a significant
improvement on videos where the camera is moving. It is also able to perform
background reconstruction on some non-video image datasets.
