  Despite the great successes achieved by deep neural networks (DNNs), recent
studies show that they are vulnerable against adversarial examples, which aim
to mislead DNNs by adding small adversarial perturbations. Several defenses
have been proposed against such attacks, while many of them have been
adaptively attacked. In this work, we aim to enhance the ML robustness from a
different perspective by leveraging domain knowledge: We propose a Knowledge
Enhanced Machine Learning Pipeline (KEMLP) to integrate domain knowledge (i.e.,
logic relationships among different predictions) into a probabilistic graphical
model via first-order logic rules. In particular, we develop KEMLP by
integrating a diverse set of weak auxiliary models based on their logical
relationships to the main DNN model that performs the target task.
Theoretically, we provide convergence results and prove that, under mild
conditions, the prediction of KEMLP is more robust than that of the main DNN
model. Empirically, we take road sign recognition as an example and leverage
the relationships between road signs and their shapes and contents as domain
knowledge. We show that compared with adversarial training and other baselines,
KEMLP achieves higher robustness against physical attacks, $\mathcal{L}_p$
bounded attacks, unforeseen attacks, and natural corruptions under both
whitebox and blackbox settings, while still maintaining high clean accuracy.
