  With the ongoing rapid adoption of Artificial Intelligence (AI)-based systems
in high-stakes domains, ensuring the trustworthiness, safety, and observability
of these systems has become crucial. It is essential to evaluate and monitor AI
systems not only for accuracy and quality-related metrics but also for
robustness, bias, security, interpretability, and other responsible AI
dimensions. We focus on large language models (LLMs) and other generative AI
models, which present additional challenges such as hallucinations, harmful and
manipulative content, and copyright infringement. In this survey article
accompanying our KDD 2024 tutorial, we highlight a wide range of harms
associated with generative AI systems, and survey state of the art approaches
(along with open challenges) to address these harms.
