  Deep Neural Networks (DNNs) have been widely used in software making
decisions impacting people's lives. However, they have been found to exhibit
severe erroneous behaviors that may lead to unfortunate outcomes. Previous work
shows that such misbehaviors often occur due to class property violations
rather than errors on a single image. Although methods for detecting such
errors have been proposed, fixing them has not been studied so far. Here, we
propose a generic method called Weighted Regularization (WR) consisting of five
concrete methods targeting the error-producing classes to fix the DNNs. In
particular, it can repair confusion error and bias error of DNN models for both
single-label and multi-label image classifications. A confusion error happens
when a given DNN model tends to confuse between two classes. Each method in WR
assigns more weights at a stage of DNN retraining or inference to mitigate the
confusion between target pair. A bias error can be fixed similarly. We evaluate
and compare the proposed methods along with baselines on six widely-used
datasets and architecture combinations. The results suggest that WR methods
have different trade-offs but under each setting at least one WR method can
greatly reduce confusion/bias errors at a very limited cost of the overall
performance.
