  Automatic assessment of dysarthria remains a highly challenging task due to
high variability in acoustic signals and the limited data. Currently, research
on the automatic assessment of dysarthria primarily focuses on two approaches:
one that utilizes expert features combined with machine learning, and the other
that employs data-driven deep learning methods to extract representations.
Research has demonstrated that expert features are effective in representing
pathological characteristics, while deep learning methods excel at uncovering
latent features. Therefore, integrating the advantages of expert features and
deep learning to construct a neural network architecture based on expert
knowledge may be beneficial for interpretability and assessment performance. In
this context, the present paper proposes a vowel graph attention network based
on audio-visual information, which effectively integrates the strengths of
expert knowledges and deep learning. Firstly, various features were combined as
inputs, including knowledge based acoustical features and deep learning based
pre-trained representations. Secondly, the graph network structure based on
vowel space theory was designed, allowing for a deep exploration of spatial
correlations among vowels. Finally, visual information was incorporated into
the model to further enhance its robustness and generalizability. The method
exhibited superior performance in regression experiments targeting Frenchay
scores compared to existing approaches.
