FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN

Antonios Anastasopoulos; Loïc Barrault; Luisa Bentivogli; Marcely Zanon Boito; Ondřej Bojar; Roldano Cattoni; Anna Currey; Georgiana Dinu; Kevin Duh; Maha Elbayad; Clara Emmanuel; Yannick Estève; Marcello Federico; Christian Federmann; Souhir Gahbiche; Hongyu Gong; Roman Grundkiewicz; Barry Haddow; Benjamin Hsu; Dávid Javorský; Věra Kloudová; Surafel M. Lakew; Xutai Ma; Prashant Mathur; Paul McNamee; Kenton Murray; Maria Nădejde; Satoshi Nakamura; Matteo Negri; Jan Niehues; Xing Niu; John Ortega; Juan Pino; Elizabeth Salesky; Jiatong Shi; Matthias Sperber; Sebastian Stüker; Katsuhito Sudoh; Marco Turchi; Yogesh Virkar; Alex Waibel; Changhan Wang; Shinji Watanabe

FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN

Antonios Anastasopoulos, Loïc Barrault, Luisa Bentivogli, Marcely Zanon Boito, Ondřej Bojar, Roldano Cattoni, Anna Currey, Georgiana Dinu, Kevin Duh, Maha Elbayad, Clara Emmanuel, Yannick Estève, Marcello Federico, Christian Federmann, Souhir Gahbiche, Hongyu Gong, Roman Grundkiewicz, Barry Haddow, Benjamin Hsu, Dávid JavorskýVěra Kloudová, Surafel M. Lakew, Xutai Ma, Prashant Mathur, Paul McNamee, Kenton Murray, Maria Nădejde, Satoshi Nakamura, Matteo Negri, Jan Niehues, Xing Niu, John Ortega, Juan Pino, Elizabeth Salesky, Jiatong Shi, Matthias Sperber, Sebastian Stüker, Katsuhito Sudoh, Marco Turchi, Yogesh Virkar, Alex Waibel, Changhan Wang, Shinji Watanabe

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

34 Citations (Scopus)

Abstract

The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved.

Original language	English
Title of host publication	IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference
Editors	Elizabeth Salesky, Marcello Federico, Marta Costa-Jussa
Publisher	Association for Computational Linguistics (ACL)
Pages	98-157
Number of pages	60
ISBN (Electronic)	9781955917414
Publication status	Published - 2022
Externally published	Yes
Event	19th International Conference on Spoken Language Translation, IWSLT 2022 - Dublin, Ireland Duration: 2022 May 26 → 2022 May 27

Publication series

Name	IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference

Conference

Conference	19th International Conference on Spoken Language Translation, IWSLT 2022
Country/Territory	Ireland
City	Dublin
Period	22/5/26 → 22/5/27

ASJC Scopus subject areas

Language and Linguistics
Human-Computer Interaction
Linguistics and Language

Cite this

Anastasopoulos, A., Barrault, L., Bentivogli, L., Boito, M. Z., Bojar, O., Cattoni, R., Currey, A., Dinu, G., Duh, K., Elbayad, M., Emmanuel, C., Estève, Y., Federico, M., Federmann, C., Gahbiche, S., Gong, H., Grundkiewicz, R., Haddow, B., Hsu, B., ... Watanabe, S. (2022). FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN. In E. Salesky, M. Federico, & M. Costa-Jussa (Eds.), IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference (pp. 98-157). (IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference). Association for Computational Linguistics (ACL).

FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN. / Anastasopoulos, Antonios; Barrault, Loïc; Bentivogli, Luisa et al.
IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference. ed. / Elizabeth Salesky; Marcello Federico; Marta Costa-Jussa. Association for Computational Linguistics (ACL), 2022. p. 98-157 (IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Anastasopoulos, A, Barrault, L, Bentivogli, L, Boito, MZ, Bojar, O, Cattoni, R, Currey, A, Dinu, G, Duh, K, Elbayad, M, Emmanuel, C, Estève, Y, Federico, M, Federmann, C, Gahbiche, S, Gong, H, Grundkiewicz, R, Haddow, B, Hsu, B, Javorský, D, Kloudová, V, Lakew, SM, Ma, X, Mathur, P, McNamee, P, Murray, K, Nădejde, M, Nakamura, S, Negri, M, Niehues, J, Niu, X, Ortega, J, Pino, J, Salesky, E, Shi, J, Sperber, M, Stüker, S, Sudoh, K, Turchi, M, Virkar, Y, Waibel, A, Wang, C & Watanabe, S 2022, FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN. in E Salesky, M Federico & M Costa-Jussa (eds), IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference. IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference, Association for Computational Linguistics (ACL), pp. 98-157, 19th International Conference on Spoken Language Translation, IWSLT 2022, Dublin, Ireland, 22/5/26.

Anastasopoulos A, Barrault L, Bentivogli L, Boito MZ, Bojar O, Cattoni R et al. FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN. In Salesky E, Federico M, Costa-Jussa M, editors, IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference. Association for Computational Linguistics (ACL). 2022. p. 98-157. (IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference).

Anastasopoulos, Antonios ; Barrault, Loïc ; Bentivogli, Luisa et al. / FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN. IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference. editor / Elizabeth Salesky ; Marcello Federico ; Marta Costa-Jussa. Association for Computational Linguistics (ACL), 2022. pp. 98-157 (IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference).

@inproceedings{b03f67930bec457fab7bd3af51e007e6,

title = "FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN",

abstract = "The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved.",

author = "Antonios Anastasopoulos and Lo{\"i}c Barrault and Luisa Bentivogli and Boito, {Marcely Zanon} and Ond{\v r}ej Bojar and Roldano Cattoni and Anna Currey and Georgiana Dinu and Kevin Duh and Maha Elbayad and Clara Emmanuel and Yannick Est{\`e}ve and Marcello Federico and Christian Federmann and Souhir Gahbiche and Hongyu Gong and Roman Grundkiewicz and Barry Haddow and Benjamin Hsu and D{\'a}vid Javorsk{\'y} and V{\v e}ra Kloudov{\'a} and Lakew, {Surafel M.} and Xutai Ma and Prashant Mathur and Paul McNamee and Kenton Murray and Maria N{\u a}dejde and Satoshi Nakamura and Matteo Negri and Jan Niehues and Xing Niu and John Ortega and Juan Pino and Elizabeth Salesky and Jiatong Shi and Matthias Sperber and Sebastian St{\"u}ker and Katsuhito Sudoh and Marco Turchi and Yogesh Virkar and Alex Waibel and Changhan Wang and Shinji Watanabe",

note = "Funding Information: We would like to thank the IWSLT 2022 sponsors and donors Apple, AppTek, AWS, Meta, Microsoft, and Zoom for supporting the human eval- uation of the shared tasks and student participants with computing credits. We would like to thank Mary Johnson, Tom Kocmi and Hitokazu Mat-sushita for their help with conducting parts of the human evaluation and providing useful comments. We are grateful to the many annotators who participated in the human evaluation and provided their feedback. We would like to thank Zhaoheng Ni, Jeff Hwang and the torchaudio team for providing a streaming ASR model for the simultaneous task. We would like to thank Justine Kao and Brian Bui for running the human evaluation for the speech-to-speech task. The creation of the reference interpretations was funded from the EU project H2020-ICT-2018-2-825460 (ELITR). Ondˇej Bojar would like to acknowledge the grant 19-26934X (NEUREM3) of the Czech Science Foundation. Funding Information: We would like to thank the IWSLT 2022 sponsors and donors Apple, AppTek, AWS, Meta, Microsoft, and Zoom for supporting the human evaluation of the shared tasks and student participants with computing credits. We would like to thank Mary Johnson, Tom Kocmi and Hitokazu Matsushita for their help with conducting parts of the human evaluation and providing useful comments. We are grateful to the many annotators who participated in the human evaluation and provided their feedback. We would like to thank Zhaoheng Ni, Jeff Hwang and the torchaudio team for providing a streaming ASR model for the simultaneous task. We would like to thank Justine Kao and Brian Bui for running the human evaluation for the speech-to-speech task. The creation of the reference interpretations was funded from the EU project H2020-ICT-2018-2-825460 (ELITR). Ond{\v r}ej Bojar would like to acknowledge the grant 19-26934X (NEUREM3) of the Czech Science Foundation. Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 19th International Conference on Spoken Language Translation, IWSLT 2022 ; Conference date: 26-05-2022 Through 27-05-2022",

year = "2022",

language = "English",

series = "IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference",

publisher = "Association for Computational Linguistics (ACL)",

pages = "98--157",

editor = "Elizabeth Salesky and Marcello Federico and Marta Costa-Jussa",

booktitle = "IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference",

}

TY - GEN

T1 - FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN

AU - Anastasopoulos, Antonios

AU - Barrault, Loïc

AU - Bentivogli, Luisa

AU - Boito, Marcely Zanon

AU - Bojar, Ondřej

AU - Cattoni, Roldano

AU - Currey, Anna

AU - Dinu, Georgiana

AU - Duh, Kevin

AU - Elbayad, Maha

AU - Emmanuel, Clara

AU - Estève, Yannick

AU - Federico, Marcello

AU - Federmann, Christian

AU - Gahbiche, Souhir

AU - Gong, Hongyu

AU - Grundkiewicz, Roman

AU - Haddow, Barry

AU - Hsu, Benjamin

AU - Javorský, Dávid

AU - Kloudová, Věra

AU - Lakew, Surafel M.

AU - Ma, Xutai

AU - Mathur, Prashant

AU - McNamee, Paul

AU - Murray, Kenton

AU - Nădejde, Maria

AU - Nakamura, Satoshi

AU - Negri, Matteo

AU - Niehues, Jan

AU - Niu, Xing

AU - Ortega, John

AU - Pino, Juan

AU - Salesky, Elizabeth

AU - Shi, Jiatong

AU - Sperber, Matthias

AU - Stüker, Sebastian

AU - Sudoh, Katsuhito

AU - Turchi, Marco

AU - Virkar, Yogesh

AU - Waibel, Alex

AU - Wang, Changhan

AU - Watanabe, Shinji

N1 - Funding Information: We would like to thank the IWSLT 2022 sponsors and donors Apple, AppTek, AWS, Meta, Microsoft, and Zoom for supporting the human eval- uation of the shared tasks and student participants with computing credits. We would like to thank Mary Johnson, Tom Kocmi and Hitokazu Mat-sushita for their help with conducting parts of the human evaluation and providing useful comments. We are grateful to the many annotators who participated in the human evaluation and provided their feedback. We would like to thank Zhaoheng Ni, Jeff Hwang and the torchaudio team for providing a streaming ASR model for the simultaneous task. We would like to thank Justine Kao and Brian Bui for running the human evaluation for the speech-to-speech task. The creation of the reference interpretations was funded from the EU project H2020-ICT-2018-2-825460 (ELITR). Ondˇej Bojar would like to acknowledge the grant 19-26934X (NEUREM3) of the Czech Science Foundation. Funding Information: We would like to thank the IWSLT 2022 sponsors and donors Apple, AppTek, AWS, Meta, Microsoft, and Zoom for supporting the human evaluation of the shared tasks and student participants with computing credits. We would like to thank Mary Johnson, Tom Kocmi and Hitokazu Matsushita for their help with conducting parts of the human evaluation and providing useful comments. We are grateful to the many annotators who participated in the human evaluation and provided their feedback. We would like to thank Zhaoheng Ni, Jeff Hwang and the torchaudio team for providing a streaming ASR model for the simultaneous task. We would like to thank Justine Kao and Brian Bui for running the human evaluation for the speech-to-speech task. The creation of the reference interpretations was funded from the EU project H2020-ICT-2018-2-825460 (ELITR). Ondřej Bojar would like to acknowledge the grant 19-26934X (NEUREM3) of the Czech Science Foundation. Publisher Copyright: © 2022 Association for Computational Linguistics.

PY - 2022

Y1 - 2022

N2 - The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved.

AB - The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved.

UR - http://www.scopus.com/inward/record.url?scp=85130469447&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85130469447&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85130469447

T3 - IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference

SP - 98

EP - 157

BT - IWSLT 2022 - 19th International Conference on Spoken Language Translation, Proceedings of the Conference

A2 - Salesky, Elizabeth

A2 - Federico, Marcello

A2 - Costa-Jussa, Marta

PB - Association for Computational Linguistics (ACL)

T2 - 19th International Conference on Spoken Language Translation, IWSLT 2022

Y2 - 26 May 2022 through 27 May 2022

ER -

FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this