  Recent advances in fine-grained representation learning leverage
local-to-global (emergent) relationships for achieving state-of-the-art
results. The relational representations relied upon by such methods, however,
are abstract. We aim to deconstruct this abstraction by expressing them as
interpretable graphs over image views. We begin by theoretically showing that
abstract relational representations are nothing but a way of recovering
transitive relationships among local views. Based on this, we design
Transitivity Recovering Decompositions (TRD), a graph-space search algorithm
that identifies interpretable equivalents of abstract emergent relationships at
both instance and class levels, and with no post-hoc computations. We
additionally show that TRD is provably robust to noisy views, with empirical
evidence also supporting this finding. The latter allows TRD to perform at par
or even better than the state-of-the-art, while being fully interpretable.
Implementation is available at https://github.com/abhrac/trd.
