  Overparametrization has become a de facto standard in machine learning.
Despite numerous efforts, our understanding of how and where
overparametrization helps model accuracy and robustness is still limited. To
this end, here we conduct an empirical investigation to systemically study and
replicate previous findings in this area, in particular the study by Madry et
al. Together with this study, our findings support the "universal law of
robustness" recently proposed by Bubeck et al. We argue that while critical for
robust perception, overparametrization may not be enough to achieve full
robustness and smarter architectures e.g. the ones implemented by the human
visual cortex) seem inevitable.
