  One primary focus in multimodal feature extraction is to find the
representations of individual modalities that are maximally correlated. As a
well-known measure of dependence, the Hirschfeld-Gebelein-R\'{e}nyi (HGR)
maximal correlation becomes an appealing objective because of its operational
meaning and desirable properties. However, the strict whitening constraints
formalized in the HGR maximal correlation limit its application. To address
this problem, this paper proposes Soft-HGR, a novel framework to extract
informative features from multiple data modalities. Specifically, our framework
prevents the "hard" whitening constraints, while simultaneously preserving the
same feature geometry as in the HGR maximal correlation. The objective of
Soft-HGR is straightforward, only involving two inner products, which
guarantees the efficiency and stability in optimization. We further generalize
the framework to handle more than two modalities and missing modalities. When
labels are partially available, we enhance the discriminative power of the
feature representations by making a semi-supervised adaptation. Empirical
evaluation implies that our approach learns more informative feature mappings
and is more efficient to optimize.
