  Over the past decade, deep neural networks have proven to be adept in image
classification tasks, often surpassing humans in terms of accuracy. However,
standard neural networks often fail to understand the concept of hierarchical
structures and dependencies among different classes for vision related tasks.
Humans on the other hand, seem to intuitively learn categories conceptually,
progressively growing from understanding high-level concepts down to granular
levels of categories. One of the issues arising from the inability of neural
networks to encode such dependencies within its learned structure is that of
subpopulation shift -- where models are queried with novel unseen classes taken
from a shifted population of the training set categories. Since the neural
network treats each class as independent from all others, it struggles to
categorize shifting populations that are dependent at higher levels of the
hierarchy. In this work, we study the aforementioned problems through the lens
of a novel conditional supervised training framework. We tackle subpopulation
shift by a structured learning procedure that incorporates hierarchical
information conditionally through labels. Furthermore, we introduce a notion of
graphical distance to model the catastrophic effect of mispredictions. We show
that learning in this structured hierarchical manner results in networks that
are more robust against subpopulation shifts, with an improvement up to 3\% in
terms of accuracy and up to 11\% in terms of graphical distance over standard
models on subpopulation shift benchmarks.
