  This study assesses deep learning models for audio classification in a
clinical setting with the constraint of small datasets reflecting real-world
prospective data collection. We analyze CNNs, including DenseNet and ConvNeXt,
alongside transformer models like ViT, SWIN, and AST, and compare them against
pre-trained audio models such as YAMNet and VGGish. Our method highlights the
benefits of pre-training on large datasets before fine-tuning on specific
clinical data. We prospectively collected two first-of-their-kind patient audio
datasets from stroke patients. We investigated various preprocessing
techniques, finding that RGB and grayscale spectrogram transformations affect
model performance differently based on the priors they learn from pre-training.
Our findings indicate CNNs can match or exceed transformer models in small
dataset contexts, with DenseNet-Contrastive and AST models showing notable
performance. This study highlights the significance of incremental marginal
gains through model selection, pre-training, and preprocessing in sound
classification; this offers valuable insights for clinical diagnostics that
rely on audio classification.
