  In mobile and IoT systems, Federated Learning (FL) is increasingly important
for effectively using data while maintaining user privacy. One key challenge in
FL is managing statistical heterogeneity, such as non-i.i.d. data, arising from
numerous clients and diverse data sources. This requires strategic cooperation,
often with clients having similar characteristics. However, we are interested
in a fundamental question: does achieving optimal cooperation necessarily
entail cooperating with the most similar clients? Typically, significant model
performance improvements are often realized not by partnering with the most
similar models, but through leveraging complementary data. Our theoretical and
empirical analyses suggest that optimal cooperation is achieved by enhancing
complementarity in feature distribution while restricting the disparity in the
correlation between features and targets. Accordingly, we introduce a novel
framework, \texttt{FedSaC}, which balances similarity and complementarity in FL
cooperation. Our framework aims to approximate an optimal cooperation network
for each client by optimizing a weighted sum of model similarity and feature
complementarity. The strength of \texttt{FedSaC} lies in its adaptability to
various levels of data heterogeneity and multimodal scenarios. Our
comprehensive unimodal and multimodal experiments demonstrate that
\texttt{FedSaC} markedly surpasses other state-of-the-art FL methods.
