  This work proposes a transformer architecture for user-level classification
of gambling addiction and depression that is trainable end-to-end. As opposed
to other methods that operate at the post level, we process a set of social
media posts from a particular individual, to make use of the interactions
between posts and eliminate label noise at the post level. We exploit the fact
that, by not injecting positional encodings, multi-head attention is
permutation invariant and we process randomly sampled sets of texts from a user
after being encoded with a modern pretrained sentence encoder (RoBERTa /
MiniLM). Moreover, our architecture is interpretable with modern feature
attribution methods and allows for automatic dataset creation by identifying
discriminating posts in a user's text-set. We perform ablation studies on
hyper-parameters and evaluate our method for the eRisk 2022 Lab on early
detection of signs of pathological gambling and early risk detection of
depression. The method proposed by our team BLUE obtained the best ERDE5 score
of 0.015, and the second-best ERDE50 score of 0.009 for pathological gambling
detection. For the early detection of depression, we obtained the second-best
ERDE50 of 0.027.
