  In this paper, we study a novel inference paradigm, termed as schema
inference, that learns to deductively infer the explainable predictions by
rebuilding the prior deep neural network (DNN) forwarding scheme, guided by the
prevalent philosophical cognitive concept of schema. We strive to reformulate
the conventional model inference pipeline into a graph matching policy that
associates the extracted visual concepts of an image with the pre-computed
scene impression, by analogy with human reasoning mechanism via impression
matching. To this end, we devise an elaborated architecture, termed as
SchemaNet, as a dedicated instantiation of the proposed schema inference
concept, that models both the visual semantics of input instances and the
learned abstract imaginations of target categories as topological relational
graphs. Meanwhile, to capture and leverage the compositional contributions of
visual semantics in a global view, we also introduce a universal Feat2Graph
scheme in SchemaNet to establish the relational graphs that contain abundant
interaction information. Both the theoretical analysis and the experimental
results on several benchmarks demonstrate that the proposed schema inference
achieves encouraging performance and meanwhile yields a clear picture of the
deductive process leading to the predictions. Our code is available at
https://github.com/zhfeing/SchemaNet-PyTorch.
