  Optimizing the performance of many objectives (instantiated by tasks or
clients) jointly with a few Pareto stationary solutions (models) is critical in
machine learning. However, previous multi-objective optimization methods often
focus on a few number of objectives and cannot scale to many objectives that
outnumber the solutions, leading to either subpar performance or ignored
objectives. We introduce Many-objective multi-solution Transport (MosT), a
framework that finds multiple diverse solutions in the Pareto front of many
objectives. Our insight is to seek multiple solutions, each performing as a
domain expert and focusing on a specific subset of objectives while
collectively covering all of them. MosT formulates the problem as a bi-level
optimization of weighted objectives for each solution, where the weights are
defined by an optimal transport between the objectives and solutions. Our
algorithm ensures convergence to Pareto stationary solutions for complementary
subsets of objectives. On a range of applications in federated learning,
multi-task learning, and mixture-of-prompt learning for LLMs, MosT distinctly
outperforms strong baselines, delivering high-quality, diverse solutions that
profile the entire Pareto frontier, thus ensuring balanced trade-offs across
many objectives.
