  A Collaborative Artificial Intelligence System (CAIS) performs actions in
collaboration with the human to achieve a common goal. CAISs can use a trained
AI model to control human-system interaction, or they can use human interaction
to dynamically learn from humans in an online fashion. In online learning with
human feedback, the AI model evolves by monitoring human interaction through
the system sensors in the learning state, and actuates the autonomous
components of the CAIS based on the learning in the operational state.
Therefore, any disruptive event affecting these sensors may affect the AI
model's ability to make accurate decisions and degrade the CAIS performance.
Consequently, it is of paramount importance for CAIS managers to be able to
automatically track the system performance to understand the resilience of the
CAIS upon such disruptive events. In this paper, we provide a new framework to
model CAIS performance when the system experiences a disruptive event. With our
framework, we introduce a model of performance evolution of CAIS. The model is
equipped with a set of measures that aim to support CAIS managers in the
decision process to achieve the required resilience of the system. We tested
our framework on a real-world case study of a robot collaborating online with
the human, when the system is experiencing a disruptive event. The case study
shows that our framework can be adopted in CAIS and integrated into the online
execution of the CAIS activities.
