Phonetic fluency of Japanese learners of English: Automatic vs native and non-native assessment

This study compared automatic assessments of L2 phonetic fluency of Japanese learners of English in read speech, with ratings by native and non-native English assessors, and considers whether the first language of the assessors affects the results. Speech data of 183 Japanese and 25 native English speakers' readings of “the North Wind and the Sun” were assessed for phonetic fluency by 16 trained assessors with different first languages (four American English, four Japanese and eight other languages). They rated segmental accuracy, prosody, fluency and nativelikeness. A subset of 97 of the speakers' data (the 25 native English speakers and 72 randomly selected Japanese speakers) was also used to develop an automatic fluency assessment system. The 97 speakers' data were reassessed by four different trained American raters. The correlation between the automatic evaluation and the four raters was 0.83. When the automatic system was then tested on the remaining original 111 speakers' data and the original 16 assessors' scores, it showed correlations of 0.62-0.67 for the American, Japanese and other language raters. The results suggest that the automatic assessment system can assess phonetic fluency of Japanese-accented English quite reliably, and that native and non-native evaluators used different phonetic cues to evaluate fluency.