Automatic scoring method considering quality and content of speech for scat Japanese speaking test

Naoko Okubo; Yuto Yamahata; Takeshi Yamada; Shingo Imai; Kenkichi Ishizuka; Takahiro Shinozaki; Ryuichi Nisimura; Shoji Makino; Nobuhiko Kitawaki

doi:10.1109/ICSDA.2012.6422460

Automatic scoring method considering quality and content of speech for scat Japanese speaking test

Naoko Okubo^*, Yuto Yamahata, Takeshi Yamada, Shingo Imai, Kenkichi Ishizuka, Takahiro Shinozaki, Ryuichi Nisimura, Shoji Makino, Nobuhiko Kitawaki

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.

Original language	English
Title of host publication	Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012
Pages	72-77
Number of pages	6
DOIs	https://doi.org/10.1109/ICSDA.2012.6422460
Publication status	Published - 2012
Externally published	Yes
Event	2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012 - Macau, China Duration: 2012 Dec 9 → 2012 Dec 12

Publication series

Name	Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012

Conference

Conference	2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012
Country/Territory	China
City	Macau
Period	12/12/9 → 12/12/12

Keywords

J-CAT
SCAT Japanese speaking test
automatic scoring
content of speech
quality of speech

ASJC Scopus subject areas

Software
Speech and Hearing

Access to Document

10.1109/ICSDA.2012.6422460

Cite this

Okubo, N., Yamahata, Y., Yamada, T., Imai, S., Ishizuka, K., Shinozaki, T., Nisimura, R., Makino, S., & Kitawaki, N. (2012). Automatic scoring method considering quality and content of speech for scat Japanese speaking test. In Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012 (pp. 72-77). [6422460] (Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012). https://doi.org/10.1109/ICSDA.2012.6422460

Automatic scoring method considering quality and content of speech for scat Japanese speaking test. / Okubo, Naoko; Yamahata, Yuto; Yamada, Takeshi et al.
Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012. 2012. p. 72-77 6422460 (Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Okubo, N, Yamahata, Y, Yamada, T, Imai, S, Ishizuka, K, Shinozaki, T, Nisimura, R, Makino, S & Kitawaki, N 2012, Automatic scoring method considering quality and content of speech for scat Japanese speaking test. in Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012., 6422460, Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012, pp. 72-77, 2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012, Macau, China, 12/12/9. https://doi.org/10.1109/ICSDA.2012.6422460

Okubo N, Yamahata Y, Yamada T, Imai S, Ishizuka K, Shinozaki T et al. Automatic scoring method considering quality and content of speech for scat Japanese speaking test. In Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012. 2012. p. 72-77. 6422460. (Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012). doi: 10.1109/ICSDA.2012.6422460

Okubo, Naoko ; Yamahata, Yuto ; Yamada, Takeshi et al. / Automatic scoring method considering quality and content of speech for scat Japanese speaking test. Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012. 2012. pp. 72-77 (Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012).

@inproceedings{58a0cde3254c463e854b1886bbea5331,

title = "Automatic scoring method considering quality and content of speech for scat Japanese speaking test",

abstract = "We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.",

keywords = "J-CAT, SCAT Japanese speaking test, automatic scoring, content of speech, quality of speech",

author = "Naoko Okubo and Yuto Yamahata and Takeshi Yamada and Shingo Imai and Kenkichi Ishizuka and Takahiro Shinozaki and Ryuichi Nisimura and Shoji Makino and Nobuhiko Kitawaki",

year = "2012",

doi = "10.1109/ICSDA.2012.6422460",

language = "English",

isbn = "9781467328104",

series = "Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012",

pages = "72--77",

booktitle = "Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012",

note = "2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012 ; Conference date: 09-12-2012 Through 12-12-2012",

}

TY - GEN

T1 - Automatic scoring method considering quality and content of speech for scat Japanese speaking test

AU - Okubo, Naoko

AU - Yamahata, Yuto

AU - Yamada, Takeshi

AU - Imai, Shingo

AU - Ishizuka, Kenkichi

AU - Shinozaki, Takahiro

AU - Nisimura, Ryuichi

AU - Makino, Shoji

AU - Kitawaki, Nobuhiko

PY - 2012

Y1 - 2012

N2 - We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.

AB - We are now developing a Japanese speaking test called SCAT, which is part of J-CAT (Japanese Computerized Adaptive Test), a free online proficiency test for Japanese language learners. In this paper, we focus on the sentence-reading-aloud task and the sentence generation task in SCAT, and propose an automatic scoring method for estimating the overall score of answer speech, which is holistically determined by language teachers according to a rating standard. In that process, teachers carefully consider different factors but do not rate the scores of them. We therefore analyze how each factor contributes to the overall score. The factors are divided into two categories: the quality of speech and the content of speech. The former includes pronunciation and intonation, and the latter representation and vocabulary. We then propose an automatic scoring method based on the analysis. Experimental results confirm that the proposed method gives relatively accurate estimates of the overall score.

KW - J-CAT

KW - SCAT Japanese speaking test

KW - automatic scoring

KW - content of speech

KW - quality of speech

UR - http://www.scopus.com/inward/record.url?scp=84874256962&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84874256962&partnerID=8YFLogxK

U2 - 10.1109/ICSDA.2012.6422460

DO - 10.1109/ICSDA.2012.6422460

M3 - Conference contribution

AN - SCOPUS:84874256962

SN - 9781467328104

T3 - Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012

SP - 72

EP - 77

BT - Proceedings of the 2012 International Conference on Speech Database and Assessments, Oriental COCOSDA 2012

T2 - 2012 15th International Conference on Speech Database and Assessments, Oriental COCOSDA 2012

Y2 - 9 December 2012 through 12 December 2012

ER -

Automatic scoring method considering quality and content of speech for scat Japanese speaking test

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this