  In this paper, we present NxtPost, a deployed user-to-post content-based
sequential recommender system for Facebook Groups. Inspired by recent advances
in NLP, we have adapted a Transformer-based model to the domain of sequential
recommendation. We explore causal masked multi-head attention that optimizes
both short and long-term user interests. From a user's past activities
validated by defined safety process, NxtPost seeks to learn a representation
for the user's dynamic content preference and to predict the next post user may
be interested in. In contrast to previous Transformer-based methods, we do not
assume that the recommendable posts have a fixed corpus. Accordingly, we use an
external item/token embedding to extend a sequence-based approach to a large
vocabulary. We achieve 49% abs. improvement in offline evaluation. As a result
of NxtPost deployment, 0.6% more users are meeting new people, engaging with
the community, sharing knowledge and getting support. The paper shares our
experience in developing a personalized sequential recommender system, lessons
deploying the model for cold start users, how to deal with freshness, and
tuning strategies to reach higher efficiency in online A/B experiments.
