  We propose a measure to compute class similarity in large-scale
classification based on prediction scores. Such measure has not been formally
pro-posed in the literature. We show how visualizing the class similarity
matrix can reveal hierarchical structures and relationships that govern the
classes. Through examples with various classifiers, we demonstrate how such
structures can help in analyzing the classification behavior and in inferring
potential corner cases. The source code for one example is available as a
notebook at https://github.com/bilalsal/blocks
