  Image recognition is a classic and common task in the computer vision field,
which has been widely applied in the past decade. Most existing methods in
literature aim to learn discriminative features from labeled images for
classification, however, they generally neglect confounders that infiltrate
into the learned features, resulting in low performances for discriminating
test images. To address this problem, we propose a Recursive Counterfactual
Deconfounding model for object recognition in both closed-set and open-set
scenarios based on counterfactual analysis, called RCD. The proposed model
consists of a factual graph and a counterfactual graph, where the relationships
among image features, model predictions, and confounders are built and updated
recursively for learning more discriminative features. It performs in a
recursive manner so that subtler counterfactual features could be learned and
eliminated progressively, and both the discriminability and generalization of
the proposed model could be improved accordingly. In addition, a negative
correlation constraint is designed for alleviating the negative effects of the
counterfactual features further at the model training stage. Extensive
experimental results on both closed-set recognition task and open-set
recognition task demonstrate that the proposed RCD model performs better than
11 state-of-the-art baselines significantly in most cases.
