  Pretrained Graph Neural Networks have been widely adopted for various
molecular property prediction tasks. Despite their ability to encode structural
and relational features of molecules, traditional fine-tuning of such
pretrained GNNs on the target task can lead to poor generalization. To address
this, we explore the adaptation of pretrained GNNs to the target task by
jointly training them with multiple auxiliary tasks. This could enable the GNNs
to learn both general and task-specific features, which may benefit the target
task. However, a major challenge is to determine the relatedness of auxiliary
tasks with the target task. To address this, we investigate multiple strategies
to measure the relevance of auxiliary tasks and integrate such tasks by
adaptively combining task gradients or by learning task weights via bi-level
optimization. Additionally, we propose a novel gradient surgery-based approach,
Rotation of Conflicting Gradients ($\mathtt{RCGrad}$), that learns to align
conflicting auxiliary task gradients through rotation. Our experiments with
state-of-the-art pretrained GNNs demonstrate the efficacy of our proposed
methods, with improvements of up to 7.7% over fine-tuning. This suggests that
incorporating auxiliary tasks along with target task fine-tuning can be an
effective way to improve the generalizability of pretrained GNNs for molecular
property prediction.
