  In this paper we address the instability issue of generative adversarial
network (GAN) by proposing a new similarity metric in unitary space of Schur
decomposition for 2D representations of audio and speech signals. We show that
encoding departure from normality computed in this vector space into the
generator optimization formulation helps to craft more comprehensive
spectrograms. We demonstrate the effectiveness of binding this metric for
enhancing stability in training with less mode collapse compared to baseline
GANs. Experimental results on subsets of UrbanSound8k and Mozilla common voice
datasets have shown considerable improvements on the quality of the generated
samples measured by the Fr\'echet inception distance. Moreover, reconstructed
signals from these samples, have achieved higher signal to noise ratio compared
to regular LS-GANs.
