  Risk scoring systems have been widely deployed in many applications, which
assign risk scores to users according to their behavior sequences. Though many
deep learning methods with sophisticated designs have achieved promising
results, the black-box nature hinders their applications due to fairness,
explainability, and compliance consideration. Rule-based systems are considered
reliable in these sensitive scenarios. However, building a rule system is
labor-intensive. Experts need to find informative statistics from user behavior
sequences, design rules based on statistics and assign weights to each rule. In
this paper, we bridge the gap between effective but black-box models and
transparent rule models. We propose a two-stage method, RuDi, that distills the
knowledge of black-box teacher models into rule-based student models. We design
a Monte Carlo tree search-based statistics generation method that can provide a
set of informative statistics in the first stage. Then statistics are composed
into logical rules with our proposed neural logical networks by mimicking the
outputs of teacher models. We evaluate RuDi on three real-world public datasets
and an industrial dataset to demonstrate its effectiveness.
