  Identifying potential social and ethical risks in emerging machine learning
(ML) models and their applications remains challenging. In this work, we
applied two well-established safety engineering frameworks (FMEA, STPA) to a
case study involving text-to-image models at three stages of the ML product
development pipeline: data processing, integration of a T2I model with other
models, and use. Results of our analysis demonstrate the safety frameworks -
both of which are not designed explicitly examine social and ethical risks -
can uncover failure and hazards that pose social and ethical risks. We
discovered a broad range of failures and hazards (i.e., functional, social, and
ethical) by analyzing interactions (i.e., between different ML models in the
product, between the ML product and user, and between development teams) and
processes (i.e., preparation of training data or workflows for using an ML
service/product). Our findings underscore the value and importance of examining
beyond an ML model in examining social and ethical risks, especially when we
have minimal information about an ML model.
