  The promise and proliferation of large-scale dynamic federated learning gives
rise to a prominent open question - is it prudent to share data or model across
nodes, if efficiency of transmission and fast knowledge transfer are the prime
objectives. This work investigates exactly that. Specifically, we study the
choices of exchanging raw data, synthetic data, or (partial) model updates
among devices. The implications of these strategies in the context of
foundational models are also examined in detail. Accordingly, we obtain key
insights about optimal data and model exchange mechanisms considering various
environments with different data distributions and dynamic device and network
connections. Across various scenarios that we considered, time-limited
knowledge transfer efficiency can differ by up to 9.08\%, thus highlighting the
importance of this work.
