  State of the art time automatic speech recognition (ASR) systems are becoming
increasingly complex and expensive for practical applications. This paper
presents the development of a high performance and low-footprint 4-bit
quantized LF-MMI trained factored time delay neural networks (TDNNs) based ASR
system on the 300-hr Switchboard corpus. A key feature of the overall system
design is to account for the fine-grained, varying performance sensitivity at
different model components to quantization errors. To this end, a set of neural
architectural compression and mixed precision quantization approaches were used
to facilitate hidden layer level auto-configuration of optimal factored TDNN
weight matrix subspace dimensionality and quantization bit-widths. The proposed
techniques were also used to produce 2-bit mixed precision quantized
Transformer language models. Experiments conducted on the Switchboard data
suggest that the proposed neural architectural compression and mixed precision
quantization techniques consistently outperform the uniform precision quantised
baseline systems of comparable bit-widths in terms of word error rate (WER). An
overall "lossless" compression ratio of 13.6 was obtained over the baseline
full precision system including both the TDNN and Transformer components while
incurring no statistically significant WER increase.
