  Identifying moving objects in a video sequence, which is produced by a static
camera, is a fundamental and critical task in many computer-vision
applications. A common approach performs background subtraction, which
identifies moving objects as the portion of a video frame that differs
significantly from a background model. A good background subtraction algorithm
has to be robust to changes in the illumination and it should avoid detecting
non-stationary background objects such as moving leaves, rain, snow, and
shadows. In addition, the internal background model should quickly respond to
changes in background such as objects that start to move or stop. We present a
new algorithm for video segmentation that processes the input video sequence as
a 3D matrix where the third axis is the time domain. Our approach identifies
the background by reducing the input dimension using the \emph{diffusion bases}
methodology. Furthermore, we describe an iterative method for extracting and
deleting the background. The algorithm has two versions and thus covers the
complete range of backgrounds: one for scenes with static backgrounds and the
other for scenes with dynamic (moving) backgrounds.
