  The average accuracy is one of major evaluation metrics for classification
systems, while the accuracy deviation is another important performance metric
used to evaluate various deep neural networks. In this paper, we present a new
ensemble-like fast deep neural network, Harmony, that can reduce the accuracy
deviation among categories without degrading overall average accuracy. Harmony
consists of three sub-models, namely, Target model, Complementary model, and
Conductor model. In Harmony, an object is classified by using either Target
model or Complementary model. Target model is a conventional classification
network for general categories, while Complementary model is a classification
network especially for weak categories that are inaccurately classified by
Target model. Conductor model is used to select one of two models. Experimental
results demonstrate that Harmony accurately classifies categories, while it
reduces the accuracy deviation among the categories.
