  Business success of companies heavily depends on the availability and
performance of their client applications. Due to modern development paradigms
such as DevOps and microservice architectural styles, applications are
decoupled into services with complex interactions and dependencies. Although
these paradigms enable individual development cycles with reduced delivery
times, they cause several challenges to manage the services in distributed
systems. One major challenge is to observe and monitor such distributed
systems. This paper provides a qualitative study to understand the challenges
and good practices in the field of observability and monitoring of distributed
systems. In 28 semi-structured interviews with software professionals we
discovered increasing complexity and dynamics in that field. Especially
observability becomes an essential prerequisite to ensure stable services and
further development of client applications. However, the participants mentioned
a discrepancy in the awareness regarding the importance of the topic, both from
the management as well as from the developer perspective. Besides technical
challenges, we identified a strong need for an organizational concept including
strategy, roles and responsibilities. Our results support practitioners in
developing and implementing systematic observability and monitoring for
distributed systems.
