  Over the course of the last decade, infrared (IR) and particularly thermal IR
imaging based face recognition has emerged as a promising complement to
conventional, visible spectrum based approaches which continue to struggle when
applied in practice. While inherently insensitive to visible spectrum
illumination changes, IR data introduces specific challenges of its own, most
notably sensitivity to factors which affect facial heat emission patterns, e.g.
emotional state, ambient temperature, and alcohol intake. In addition, facial
expression and pose changes are more difficult to correct in IR images because
they are less rich in high frequency detail which is an important cue for
fitting any deformable model. In this paper we describe a novel method which
addresses these major challenges. Specifically, when comparing two thermal IR
images of faces, we mutually normalize their poses and facial expressions by
using an active appearance model (AAM) to generate synthetic images of the two
faces with a neutral facial expression and in the same view (the average of the
two input views). This is achieved by piecewise affine warping which follows
AAM fitting. A major contribution of our work is the use of an AAM ensemble in
which each AAM is specialized to a particular range of poses and a particular
region of the thermal IR face space. Combined with the contributions from our
previous work which addressed the problem of reliable AAM fitting in the
thermal IR spectrum, and the development of a person-specific representation
robust to transient changes in the pattern of facial temperature emissions, the
proposed ensemble framework accurately matches faces across the full range of
yaw from frontal to profile, even in the presence of scale variation (e.g. due
to the varying distance of a subject from the camera).
