  The rapid expansion of the Internet of Things (IoT) and Edge Computing has
presented challenges for centralized Machine and Deep Learning (ML/DL) methods
due to the presence of distributed data silos that hold sensitive information.
To address concerns regarding data privacy, collaborative and
privacy-preserving ML/DL techniques like Federated Learning (FL) have emerged.
However, ensuring data privacy and performance alone is insufficient since
there is a growing need to establish trust in model predictions. Existing
literature has proposed various approaches on trustworthy ML/DL (excluding data
privacy), identifying robustness, fairness, explainability, and accountability
as important pillars. Nevertheless, further research is required to identify
trustworthiness pillars and evaluation metrics specifically relevant to FL
models, as well as to develop solutions that can compute the trustworthiness
level of FL models. This work examines the existing requirements for evaluating
trustworthiness in FL and introduces a comprehensive taxonomy consisting of six
pillars (privacy, robustness, fairness, explainability, accountability, and
federation), along with over 30 metrics for computing the trustworthiness of FL
models. Subsequently, an algorithm named FederatedTrust is designed based on
the pillars and metrics identified in the taxonomy to compute the
trustworthiness score of FL models. A prototype of FederatedTrust is
implemented and integrated into the learning process of FederatedScope, a
well-established FL framework. Finally, five experiments are conducted using
different configurations of FederatedScope to demonstrate the utility of
FederatedTrust in computing the trustworthiness of FL models. Three experiments
employ the FEMNIST dataset, and two utilize the N-BaIoT dataset considering a
real-world IoT security use case.
