  To design with AI models, user experience (UX) designers must assess the fit
between the model and user needs. Based on user research, they need to
contextualize the model's behavior and potential failures within their
product-specific data instances and user scenarios. However, our formative
interviews with ten UX professionals revealed that such a proactive discovery
of model limitations is challenging and time-intensive. Furthermore, designers
often lack technical knowledge of AI and accessible exploration tools, which
challenges their understanding of model capabilities and limitations. In this
work, we introduced a failure-driven design approach to AI, a workflow that
encourages designers to explore model behavior and failure patterns early in
the design process. The implementation of fAIlureNotes, a designer-centered
failure exploration and analysis tool, supports designers in evaluating models
and identifying failures across diverse user groups and scenarios. Our
evaluation with UX practitioners shows that fAIlureNotes outperforms today's
interactive model cards in assessing context-specific model performance.
