  Related Pins is the Web-scale recommender system that powers over 40% of user
engagement on Pinterest. This paper is a longitudinal study of three years of
its development, exploring the evolution of the system and its components from
prototypes to present state. Each component was originally built with many
constraints on engineering effort and computational resources, so we
prioritized the simplest and highest-leverage solutions. We show how organic
growth led to a complex system and how we managed this complexity. Many
challenges arose while building this system, such as avoiding feedback loops,
evaluating performance, activating content, and eliminating legacy heuristics.
Finally, we offer suggestions for tackling these challenges when engineering
Web-scale recommender systems.
