  Factorization machines (FM) are a popular model class to learn pairwise
interactions by a low-rank approximation. Different from existing FM-based
approaches which use a fixed rank for all features, this paper proposes a
Rank-Aware FM (RaFM) model which adopts pairwise interactions from embeddings
with different ranks. The proposed model achieves a better performance on
real-world datasets where different features have significantly varying
frequencies of occurrences. Moreover, we prove that the RaFM model can be
stored, evaluated, and trained as efficiently as one single FM, and under some
reasonable conditions it can be even significantly more efficient than FM. RaFM
improves the performance of FMs in both regression tasks and classification
tasks while incurring less computational burden, therefore also has attractive
potential in industrial applications.
