  Job marketplace is a heterogeneous graph composed of interactions among
members (job-seekers), companies, and jobs. Understanding and modeling job
marketplace can benefit both job seekers and employers, ultimately contributing
to the greater good of the society. However, existing graph neural network
(GNN)-based methods have shallow understandings of the associated textual
features and heterogeneous relations. To address the above challenges, we
propose PLM4Job, a job marketplace foundation model that tightly couples
pretrained language models (PLM) with job market graph, aiming to fully utilize
the pretrained knowledge and reasoning ability to model member/job textual
features as well as various member-job relations simultaneously. In the
pretraining phase, we propose a heterogeneous ego-graph-based prompting
strategy to model and aggregate member/job textual features based on the
topological structure around the target member/job node, where entity type
embeddings and graph positional embeddings are introduced accordingly to model
different entities and their heterogeneous relations. Meanwhile, a
proximity-aware attention alignment strategy is designed to dynamically adjust
the attention of the PLM on ego-graph node tokens in the prompt, such that the
attention can be better aligned with job marketplace semantics. Extensive
experiments at LinkedIn demonstrate the effectiveness of PLM4Job.
