  We study the impact of personalized content recommendations on the usage of
an educational app for children. In a randomized controlled trial, we show that
the introduction of personalized recommendations increases the consumption of
content in the personalized section of the app by approximately 60%. We further
show that the overall app usage increases by 14%, compared to the baseline
system where human content editors select stories for all students at a given
grade level. The magnitude of individual gains from personalized content
increases with the amount of data available about a student and with
preferences for niche content: heavy users with long histories of content
interactions who prefer niche content benefit more than infrequent, newer users
who like popular content. To facilitate the move to personalized recommendation
systems from a simpler system, we describe how we make important design
decisions, such as comparing alternative models using offline metrics and
choosing the right target audience.
