  Pre-trained language models consider the context of neighboring words and
documents but lack any author context of the human generating the text.
However, language depends on the author's states, traits, social, situational,
and environmental attributes, collectively referred to as human context (Soni
et al., 2024). Human-centered natural language processing requires
incorporating human context into language models. Currently, two methods exist:
pre-training with 1) group-wise attributes (e.g., over-45-year-olds) or 2)
individual traits. Group attributes are simple but coarse -- not all
45-year-olds write the same way -- while individual traits allow for more
personalized representations, but require more complex modeling and data. It is
unclear which approach benefits what tasks. We compare pre-training models with
human context via 1) group attributes, 2) individual users, and 3) a combined
approach on five user- and document-level tasks. Our results show that there is
no best approach, but that human-centered language modeling holds avenues for
different methods.
