  Latent factor models are the driving forces of the state-of-the-art
recommender systems, with an important insight of vectorizing raw input
features into dense embeddings. The dimensions of different feature embeddings
are often set to a same value empirically, which limits the predictive
performance of latent factor models. Existing works have proposed heuristic or
reinforcement learning-based methods to search for mixed feature embedding
dimensions. For efficiency concern, these methods typically choose embedding
dimensions from a restricted set of candidate dimensions. However, this
restriction will hurt the flexibility of dimension selection, leading to
suboptimal performance of search results. In this paper, we propose
Differentiable Neural Input Search (DNIS), a method that searches for mixed
feature embedding dimensions in a more flexible space through continuous
relaxation and differentiable optimization. The key idea is to introduce a soft
selection layer that controls the significance of each embedding dimension, and
optimize this layer according to model's validation performance. DNIS is
model-agnostic and thus can be seamlessly incorporated with existing latent
factor models for recommendation. We conduct experiments with various
architectures of latent factor models on three public real-world datasets for
rating prediction, Click-Through-Rate (CTR) prediction, and top-k item
recommendation. The results demonstrate that our method achieves the best
predictive performance compared with existing neural input search approaches
with fewer embedding parameters and less time cost.
