  We present a scalable and efficient neural waveform coding system for speech
compression. We formulate the speech coding problem as an autoencoding task,
where a convolutional neural network (CNN) performs encoding and decoding as a
neural waveform codec (NWC) during its feedforward routine. The proposed NWC
also defines quantization and entropy coding as a trainable module, so the
coding artifacts and bitrate control are handled during the optimization
process. We achieve efficiency by introducing compact model components to NWC,
such as gated residual networks and depthwise separable convolution.
Furthermore, the proposed models are with a scalable architecture, cross-module
residual learning (CMRL), to cover a wide range of bitrates. To this end, we
employ the residual coding concept to concatenate multiple NWC autoencoding
modules, where each NWC module performs residual coding to restore any
reconstruction loss that its preceding modules have created. CMRL can scale
down to cover lower bitrates as well, for which it employs linear predictive
coding (LPC) module as its first autoencoder. The hybrid design integrates LPC
and NWC by redefining LPC's quantization as a differentiable process, making
the system training an end-to-end manner. The decoder of proposed system is
with either one NWC (0.12 million parameters) in low to medium bitrate ranges
(12 to 20 kbps) or two NWCs in the high bitrate (32 kbps). Although the
decoding complexity is not yet as low as that of conventional speech codecs, it
is significantly reduced from that of other neural speech coders, such as a
WaveNet-based vocoder. For wide-band speech coding quality, our system yields
comparable or superior performance to AMR-WB and Opus on TIMIT test utterances
at low and medium bitrates. The proposed system can scale up to higher bitrates
to achieve near transparent performance.
