  Modern recommender systems operate in a fully server-based fashion. To cater
to millions of users, the frequent model maintaining and the high-speed
processing for concurrent user requests are required, which comes at the cost
of a huge carbon footprint. Meanwhile, users need to upload their behavior data
even including the immediate environmental context to the server, raising the
public concern about privacy. On-device recommender systems circumvent these
two issues with cost-conscious settings and local inference. However, due to
the limited memory and computing resources, on-device recommender systems are
confronted with two fundamental challenges: (1) how to reduce the size of
regular models to fit edge devices? (2) how to retain the original capacity?
Previous research mostly adopts tensor decomposition techniques to compress the
regular recommendation model with limited compression ratio so as to avoid
drastic performance degradation. In this paper, we explore ultra-compact models
for next-item recommendation, by loosing the constraint of dimensionality
consistency in tensor decomposition. Meanwhile, to compensate for the capacity
loss caused by compression, we develop a self-supervised knowledge distillation
framework which enables the compressed model (student) to distill the essential
information lying in the raw data, and improves the long-tail item
recommendation through an embedding-recombination strategy with the original
model (teacher). The extensive experiments on two benchmarks demonstrate that,
with 30x model size reduction, the compressed model almost comes with no
accuracy loss, and even outperforms its uncompressed counterpart in most cases.
