  The loss landscape of deep neural networks (DNNs) is commonly considered
complex and wildly fluctuated. However, an interesting observation is that the
loss surfaces plotted along Gaussian noise directions are almost v-basin ones
with the perturbed model lying on the basin. This motivates us to rethink
whether the 1D or 2D subspace could cover more complex local geometry
structures, and how to mine the corresponding perturbation directions. This
paper systematically and gradually categorizes the 1D curves from simple to
complex, including v-basin, v-side, w-basin, w-peak, and vvv-basin curves.
Notably, the latter two types are already hard to obtain via the intuitive
construction of specific perturbation directions, and we need to propose proper
mining algorithms to plot the corresponding 1D curves. Combining these 1D
directions, various types of 2D surfaces are visualized such as the saddle
surfaces and the bottom of a bottle of wine that are only shown by demo
functions in previous works. Finally, we propose theoretical insights from the
lens of the Hessian matrix to explain the observed several interesting
phenomena.
