  In recent years, there has been a growing interest in visualizing the loss
landscape of neural networks. Linear landscape visualization methods, such as
principal component analysis, have become widely used as they intuitively help
researchers study neural networks and their training process. However, these
linear methods suffer from limitations and drawbacks due to their lack of
flexibility and low fidelity at representing the high dimensional landscape. In
this paper, we present a novel auto-encoder-based non-linear landscape
visualization method called Neuro-Visualizer that addresses these shortcoming
and provides useful insights about neural network loss landscapes. To
demonstrate its potential, we run experiments on a variety of problems in two
separate applications of knowledge-guided machine learning (KGML). Our findings
show that Neuro-Visualizer outperforms other linear and non-linear baselines
and helps corroborate, and sometime challenge, claims proposed by machine
learning community. All code and data used in the experiments of this paper are
available at an anonymous link
https://anonymous.4open.science/r/NeuroVisualizer-FDD6
