  User modeling is crucial to understanding user behavior and essential for
improving user experience and personalized recommendations. When users interact
with software, vast amounts of command sequences are generated through logging
and analytics systems. These command sequences contain clues to the users'
goals and intents. However, these data modalities are highly unstructured and
unlabeled, making it difficult for standard predictive systems to learn from.
We propose SimCURL, a simple yet effective contrastive self-supervised deep
learning framework that learns user representation from unlabeled command
sequences. Our method introduces a user-session network architecture, as well
as session dropout as a novel way of data augmentation. We train and evaluate
our method on a real-world command sequence dataset of more than half a billion
commands. Our method shows significant improvement over existing methods when
the learned representation is transferred to downstream tasks such as
experience and expertise classification.
