  Learning general-purpose user representations based on user behavioral logs
is an increasingly popular user modeling approach. It benefits from easily
available, privacy-friendly yet expressive data, and does not require extensive
re-tuning of the upstream user model for different downstream tasks. While this
approach has shown promise in search engines and e-commerce applications, its
fit for instant messaging platforms, a cornerstone of modern digital
communication, remains largely uncharted. We explore this research gap using
Snapchat data as a case study. Specifically, we implement a Transformer-based
user model with customized training objectives and show that the model can
produce high-quality user representations across a broad range of evaluation
tasks, among which we introduce three new downstream tasks that concern pivotal
topics in user research: user safety, engagement and churn. We also tackle the
challenge of efficient extrapolation of long sequences at inference time, by
applying a novel positional encoding method.
