  Automatic Speech Assessment (ASA) has seen notable advancements with the
utilization of self-supervised features (SSL) in recent research. However, a
key challenge in ASA lies in the imbalanced distribution of data, particularly
evident in English test datasets. To address this challenge, we approach ASA as
an ordinal classification task, introducing Weighted Vectors Ranking Similarity
(W-RankSim) as a novel regularization technique. W-RankSim encourages closer
proximity of weighted vectors in the output layer for similar classes, implying
that feature vectors with similar labels would be gradually nudged closer to
each other as they converge towards corresponding weighted vectors. Extensive
experimental evaluations confirm the effectiveness of our approach in improving
ordinal classification performance for ASA. Furthermore, we propose a hybrid
model that combines SSL and handcrafted features, showcasing how the inclusion
of handcrafted features enhances performance in an ASA system.
