  We propose a novel training methodology -- Concept Group Learning (CGL) --
that encourages training of interpretable CNN filters by partitioning filters
in each layer into concept groups, each of which is trained to learn a single
visual concept. We achieve this through a novel regularization strategy that
forces filters in the same group to be active in similar image regions for a
given layer. We additionally use a regularizer to encourage a sparse weighting
of the concept groups in each layer so that a few concept groups can have
greater importance than others. We quantitatively evaluate CGL's model
interpretability using standard interpretability evaluation techniques and find
that our method increases interpretability scores in most cases. Qualitatively
we compare the image regions that are most active under filters learned using
CGL versus filters learned without CGL and find that CGL activation regions
more strongly concentrate around semantically relevant features.
