  Automatic pronunciation evaluation plays an important role in pronunciation
training and second language education. This field draws heavily on concepts
from automatic speech recognition (ASR) to quantify how close the pronunciation
of non-native speech is to native-like pronunciation. However, it is known that
the formation of accent is related to pronunciation patterns of both the target
language (L2) and the speaker's first language (L1). In this paper, we propose
to use two native speech acoustic models, one trained on L2 speech and the
other trained on L1 speech. We develop two sets of measurements that can be
extracted from two acoustic models given accented speech. A new utterance-level
feature extraction scheme is used to convert these measurements into a
fixed-dimension vector which is used as an input to a statistical model to
predict the accentedness of a speaker. On a data set consisting of speakers
from 4 different L1 backgrounds, we show that the proposed system yields
improved correlation with human evaluators compared to systems only using the
L2 acoustic model.
