  This paper describes a novel energy-based probabilistic distribution that
represents complex-valued data and explains how to apply it to direct feature
extraction from complex-valued spectra. The proposed model, the complex-valued
restricted Boltzmann machine (CRBM), is designed to deal with complex-valued
visible units as an extension of the well-known restricted Boltzmann machine
(RBM). Like the RBM, the CRBM learns the relationships between visible and
hidden units without having connections between units in the same layer, which
dramatically improves training efficiency by using Gibbs sampling or
contrastive divergence (CD). Another important characteristic is that the CRBM
also has connections between real and imaginary parts of each of the
complex-valued visible units that help represent the data distribution in the
complex domain. In speech signal processing, classification and generation
features are often based on amplitude spectra (e.g., MFCC, cepstra, and
mel-cepstra) even if they are calculated from complex spectra, and they ignore
phase information. In contrast, the proposed feature extractor using the CRBM
directly encodes the complex spectra (or another complex-valued representation
of the complex spectra) into binary-valued latent features (hidden units).
Since the visible-hidden connections are undirected, we can also recover
(decode) the complex spectra from the latent features directly. Our speech
coding experiments demonstrated that the CRBM outperformed other speech coding
methods, such as methods using the conventional RBM, the mel-log spectrum
approximate (MLSA) decoder, etc.
