  In this paper, we generate and control semantically interpretable filters
that are directly learned from natural images in an unsupervised fashion. Each
semantic filter learns a visually interpretable local structure in conjunction
with other filters. The significance of learning these interpretable filter
sets is demonstrated on two contrasting applications. The first application is
image recognition under progressive decolorization, in which recognition
algorithms should be color-insensitive to achieve a robust performance. The
second application is image quality assessment where objective methods should
be sensitive to color degradations. In the proposed work, the sensitivity and
lack thereof are controlled by weighing the semantic filters based on the local
structures they represent. To validate the proposed approach, we utilize the
CURE-TSR dataset for image recognition and the TID 2013 dataset for image
quality assessment. We show that the proposed semantic filter set achieves
state-of-the-art performances in both datasets while maintaining its robustness
across progressive distortions.
