  The opacity of AI models necessitates both validation and evaluation before
their integration into services. To investigate these models, explainable AI
(XAI) employs methods that elucidate the relationship between input features
and output predictions. The operations of XAI extend beyond the execution of a
single algorithm, involving a series of activities that include preprocessing
data, adjusting XAI to align with model parameters, invoking the model to
generate predictions, and summarizing the XAI results. Adversarial attacks are
well-known threats that aim to mislead AI models. The assessment complexity,
especially for XAI, increases when open-source AI models are subject to
adversarial attacks, due to various combinations. To automate the numerous
entities and tasks involved in XAI-based assessments, we propose a cloud-based
service framework that encapsulates computing components as microservices and
organizes assessment tasks into pipelines. The current XAI tools are not
inherently service-oriented. This framework also integrates open XAI tool
libraries as part of the pipeline composition. We demonstrate the application
of XAI services for assessing five quality attributes of AI models: (1)
computational cost, (2) performance, (3) robustness, (4) explanation deviation,
and (5) explanation resilience across computer vision and tabular cases. The
service framework generates aggregated analysis that showcases the quality
attributes for more than a hundred combination scenarios.
