  This work introduces BRILLsson, a novel binary neural network-based
representation learning model for a broad range of non-semantic speech tasks.
We train the model with knowledge distillation from a large and real-valued
TRILLsson model with only a fraction of the dataset used to train TRILLsson.
The resulting BRILLsson models are only 2MB in size with a latency less than
8ms, making them suitable for deployment in low-resource devices such as
wearables. We evaluate BRILLsson on eight benchmark tasks (including but not
limited to spoken language identification, emotion recognition, health
condition diagnosis, and keyword spotting), and demonstrate that our proposed
ultra-light and low-latency models perform as well as large-scale models.
