  Nowadays Artificial Intelligence (AI) has become a fundamental component of
healthcare applications, both clinical and remote, but the best performing AI
systems are often too complex to be self-explaining. Explainable AI (XAI)
techniques are defined to unveil the reasoning behind the system's predictions
and decisions, and they become even more critical when dealing with sensitive
and personal health data. It is worth noting that XAI has not gathered the same
attention across different research areas and data types, especially in
healthcare. In particular, many clinical and remote health applications are
based on tabular and time series data, respectively, and XAI is not commonly
analysed on these data types, while computer vision and Natural Language
Processing (NLP) are the reference applications. To provide an overview of XAI
methods that are most suitable for tabular and time series data in the
healthcare domain, this paper provides a review of the literature in the last 5
years, illustrating the type of generated explanations and the efforts provided
to evaluate their relevance and quality. Specifically, we identify clinical
validation, consistency assessment, objective and standardised quality
evaluation, and human-centered quality assessment as key features to ensure
effective explanations for the end users. Finally, we highlight the main
research challenges in the field as well as the limitations of existing XAI
methods.
