  Spectrogram classification plays an important role in analyzing gravitational
wave data. In this paper, we propose a framework to improve the classification
performance by using Generative Adversarial Networks (GANs). As substantial
efforts and expertise are required to annotate spectrograms, the number of
training examples is very limited. However, it is well known that deep networks
can perform well only when the sample size of the training set is sufficiently
large. Furthermore, the imbalanced sample sizes in different classes can also
hamper the performance. In order to tackle these problems, we propose a
GAN-based data augmentation framework. While standard data augmentation methods
for conventional images cannot be applied on spectrograms, we found that a
variant of GANs, ProGAN, is capable of generating high-resolution spectrograms
which are consistent with the quality of the high-resolution original images
and provide a desirable diversity. We have validated our framework by
classifying glitches in the {\it Gravity Spy} dataset with the GAN-generated
spectrograms for training. We show that the proposed method can provide an
alternative to transfer learning for the classification of spectrograms using
deep networks, i.e. using a high-resolution GAN for data augmentation instead.
Furthermore, fluctuations in classification performance with small sample sizes
for training and evaluation can be greatly reduced. Using the trained network
in our framework, we have also examined the spectrograms with label anomalies
in {\it Gravity Spy}.
