  The generalization ability of Deep Neural Networks (DNNs) is still not fully
understood, despite numerous theoretical and empirical analyses. Recently,
Allen-Zhu & Li (2023) introduced the concept of multi-views to explain the
generalization ability of DNNs, but their main target is ensemble or distilled
models, and no method for estimating multi-views used in a prediction of a
specific input is discussed. In this paper, we propose Minimal Sufficient Views
(MSVs), which is similar to multi-views but can be efficiently computed for
real images. MSVs is a set of minimal and distinct features in an input, each
of which preserves a model's prediction for the input. We empirically show that
there is a clear relationship between the number of MSVs and prediction
accuracy across models, including convolutional and transformer models,
suggesting that a multi-view like perspective is also important for
understanding the generalization ability of (non-ensemble or non-distilled)
DNNs.
