  Fixed-point (FXP) inference has proven suitable for embedded devices with
limited computational resources, and yet model training is continually
performed in floating-point (FLP). FXP training has not been fully explored and
the non-trivial conversion from FLP to FXP presents unavoidable performance
drop. We propose a novel method to train and obtain FXP convolutional
keyword-spotting (KWS) models. We combine our methodology with two
quantization-aware-training (QAT) techniques - squashed weight distribution and
absolute cosine regularization for model parameters, and propose techniques for
extending QAT over transient variables, otherwise neglected by previous
paradigms. Experimental results on the Google Speech Commands v2 dataset show
that we can reduce model precision up to 4-bit with no loss in accuracy.
Furthermore, on an in-house KWS dataset, we show that our 8-bit FXP-QAT models
have a 4-6% improvement in relative false discovery rate at fixed false reject
rate compared to full precision FLP models. During inference we argue that
FXP-QAT eliminates q-format normalization and enables the use of low-bit
accumulators while maximizing SIMD throughput to reduce user perceived latency.
We demonstrate that we can reduce execution time by 68% without compromising
KWS model's predictive performance or requiring model architectural changes.
Our work provides novel findings that aid future research in this area and
enable accurate and efficient models.
