  To reduce the need for skilled clinicians in heart sound interpretation,
recent studies on automating cardiac auscultation have explored deep learning
approaches. However, despite the demands for large data for deep learning, the
size of the heart sound datasets is limited, and no pre-trained model is
available. On the contrary, many pre-trained models for general audio tasks are
available as general-purpose audio representations. This study explores the
potential of general-purpose audio representations pre-trained on large-scale
datasets for transfer learning in heart murmur detection. Experiments on the
CirCor DigiScope heart sound dataset show that the recent self-supervised
learning Masked Modeling Duo (M2D) outperforms previous methods with the
results of a weighted accuracy of 0.832 and an unweighted average recall of
0.713. Experiments further confirm improved performance by ensembling M2D with
other models. These results demonstrate the effectiveness of general-purpose
audio representation in processing heart sounds and open the way for further
applications. Our code is available online which runs on a 24 GB consumer GPU
at https://github.com/nttcslab/m2d/tree/master/app/circor
