  Embedding learning is an important technique in deep recommendation models to
map categorical features to dense vectors. However, the embedding tables often
demand an extremely large number of parameters, which become the storage and
efficiency bottlenecks. Distributed training solutions have been adopted to
partition the embedding tables into multiple devices. However, the embedding
tables can easily lead to imbalances if not carefully partitioned. This is a
significant design challenge of distributed systems named embedding table
sharding, i.e., how we should partition the embedding tables to balance the
costs across devices, which is a non-trivial task because 1) it is hard to
efficiently and precisely measure the cost, and 2) the partition problem is
known to be NP-hard. In this work, we introduce our novel practice in Meta,
namely AutoShard, which uses a neural cost model to directly predict the
multi-table costs and leverages deep reinforcement learning to solve the
partition problem. Experimental results on an open-sourced large-scale
synthetic dataset and Meta's production dataset demonstrate the superiority of
AutoShard over the heuristics. Moreover, the learned policy of AutoShard can
transfer to sharding tasks with various numbers of tables and different ratios
of the unseen tables without any fine-tuning. Furthermore, AutoShard can
efficiently shard hundreds of tables in seconds. The effectiveness,
transferability, and efficiency of AutoShard make it desirable for production
use. Our algorithms have been deployed in Meta production environment. A
prototype is available at https://github.com/daochenzha/autoshard
