  With the growing adoption of machine learning techniques, there is a surge of
research interest towards making machine learning systems more transparent and
interpretable. Various visualizations have been developed to help model
developers understand, diagnose, and refine machine learning models. However, a
large number of potential but neglected users are the domain experts with
little knowledge of machine learning but are expected to work with machine
learning systems. In this paper, we present an interactive visualization
technique to help users with little expertise in machine learning to
understand, explore and validate predictive models. By viewing the model as a
black box, we extract a standardized rule-based knowledge representation from
its input-output behavior. We design RuleMatrix, a matrix-based visualization
of rules to help users navigate and verify the rules and the black-box model.
We evaluate the effectiveness of RuleMatrix via two use cases and a usability
study.
