The problem of multi-sensors image sequences registra-tion requires
selecting features that are invariant across sensors for a
simultaneous alignment in spatial and tempo-ral domain. We use
gradient information for extracting consistent features across
In most of the cases, the temporal constraint is assumed to be known,
and the spatial constraint is assumed to be constant. However, in this
study, we did not have access to the calibration information of the
rig and we had to register the geometrically the sensors using image
features. Al-though the sensors are tightly mounted on rigid structure
(e.g. camera rig), spatial variation can be observed frame by frame
due to small errors in of the estimated perspective projection
registering the views.
The geometric registration of the sensors is performed by a
combination of a perspective and affine transforma-tions using the set
of extracted invariant features. The per-spective transformation is
estimated from the first frame of each sensor, and is used as an
initial registration. The affine transformation is estimated from
pairs of frames across sensors to estimate spatial drift between
sensors. The opti-mal transformation is selected by measuring
registration errors from both transformations.
The purpose of the sensor registration is the detection and tracking
of the moving objects in the scene.