  The ever-growing concerns regarding data privacy have led to a paradigm shift
in machine learning (ML) architectures from centralized to distributed
approaches, giving rise to federated learning (FL) and split learning (SL) as
the two predominant privacy-preserving ML mechanisms. However,implementing FL
or SL in device-to-device (D2D)-enabled heterogeneous networks with diverse
clients presents substantial challenges, including architecture scalability and
prolonged training delays. To address these challenges, this article introduces
two innovative hybrid distributed ML architectures, namely, hybrid split FL
(HSFL) and hybrid federated SL (HFSL). Such architectures combine the strengths
of both FL and SL in D2D-enabled heterogeneous wireless networks. We provide a
comprehensive analysis of the performance and advantages of HSFL and HFSL,
while also highlighting open challenges for future exploration. We support our
proposals with preliminary simulations using three datasets in non-independent
and non-identically distributed settings, demonstrating the feasibility of our
architectures. Our simulations reveal notable reductions in
communication/computation costs and training delays as compared to conventional
FL and SL.
