  The problem of multimodal clustering arises whenever the data are gathered
with several physically different sensors. Observations from different
modalities are not necessarily aligned in the sense there there is no obvious
way to associate or to compare them in some common space. A solution may
consist in considering multiple clustering tasks independently for each
modality. The main difficulty with such an approach is to guarantee that the
unimodal clusterings are mutually consistent. In this paper we show that
multimodal clustering can be addressed within a novel framework, namely
conjugate mixture models. These models exploit the explicit transformations
that are often available between an unobserved parameter space (objects) and
each one of the observation spaces (sensors). We formulate the problem as a
likelihood maximization task and we derive the associated conjugate
expectation-maximization algorithm. The convergence properties of the proposed
algorithm are thoroughly investigated. Several local/global optimization
techniques are proposed in order to increase its convergence speed. Two
initialization strategies are proposed and compared. A consistent
model-selection criterion is proposed. The algorithm and its variants are
tested and evaluated within the task of 3D localization of several speakers
using both auditory and visual data.
