  In modern recommender system applications, such as e-commerce, predicting
multiple targets like click-through rate (CTR) and post-view click-through \&
conversion rate (CTCVR) is common. Multi-task recommender systems are gaining
traction in research and practical use. Existing multi-task recommender systems
tackle diverse business scenarios, merging and modeling these scenarios unlocks
shared knowledge to boost overall performance. As new and more complex
real-world recommendation scenarios have emerged, data privacy issues make it
difficult to train a single global multi-task recommendation model that
processes multiple separate scenarios.
  In this paper, we propose a novel framework for personalized federated
multi-scenario multi-task recommendation, called PF-MSMTrec. We assign each
scenario to a dedicated client, with each client utilizing the
Mixture-of-Experts (MMoE) structure. Our proposed method aims to tackle the
unique challenge posed by multiple optimization conflicts in this setting. We
introduce a bottom-up joint learning mechanism. Firstly, we design a parameter
template to decouple the parameters of the expert network. Thus, scenario
parameters are shared knowledge for federated parameter aggregation, while
task-specific parameters are personalized local parameters. Secondly, we
conduct personalized federated learning for the parameters of each expert
network through a federated communication round, utilizing three modules:
federated batch normalization, conflict coordination, and personalized
aggregation. Finally, we perform another round of personalized federated
parameter aggregation on the task tower network to obtain the prediction
results for multiple tasks. We conduct extensive experiments on two public
datasets, and the results demonstrate that our proposed method surpasses
state-of-the-art methods.
