Multilingual End-To-End Speech Translation

Hirofumi Inaguma; Kevin Duh; Tatsuya Kawahara; Shinji Watanabe

doi:10.1109/ASRU46091.2019.9003832

Multilingual End-To-End Speech Translation

Hirofumi Inaguma, Kevin Duh, Tatsuya Kawahara, Shinji Watanabe

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

38 Citations (Scopus)

Abstract

In this paper, we propose a simple yet effective framework for multilingual end-To-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-To-sequence architecture. While multilingual models have shown to be useful for automatic speech recognition (ASR) and machine translation (MT), this is the first time they are applied to the end-To-end ST problem. We show the effectiveness of multilingual end-To-end ST in two scenarios: one-To-many and many-To-many translations with publicly available data. We experimentally confirm that multilingual end-To-end ST models significantly outperform bilingual ones in both scenarios. The generalization of multilingual training is also evaluated in a transfer learning scenario to a very low-resource language pair. All of our codes and the database are publicly available to encourage further research in this emergent multilingual ST topic¹¹Available at https://github.com/espnet/espnet.

Original language	English
Title of host publication	2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	570-577
Number of pages	8
ISBN (Electronic)	9781728103068
DOIs	https://doi.org/10.1109/ASRU46091.2019.9003832
Publication status	Published - 2019 Dec
Externally published	Yes
Event	2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Singapore, Singapore Duration: 2019 Dec 15 → 2019 Dec 18

Publication series

Name	2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings

Conference

Conference	2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019
Country/Territory	Singapore
City	Singapore
Period	19/12/15 → 19/12/18

Keywords

Speech translation
attention-based sequence-To-sequence
multilingual end-To-end speech translation
transfer learning

ASJC Scopus subject areas

Computer Networks and Communications
Signal Processing
Linguistics and Language
Communication

Access to Document

10.1109/ASRU46091.2019.9003832

Cite this

Inaguma, H., Duh, K., Kawahara, T., & Watanabe, S. (2019). Multilingual End-To-End Speech Translation. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings (pp. 570-577). [9003832] (2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ASRU46091.2019.9003832

Multilingual End-To-End Speech Translation. / Inaguma, Hirofumi; Duh, Kevin; Kawahara, Tatsuya et al.
2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2019. p. 570-577 9003832 (2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Inaguma, H, Duh, K, Kawahara, T & Watanabe, S 2019, Multilingual End-To-End Speech Translation. in 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings., 9003832, 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 570-577, 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019, Singapore, Singapore, 19/12/15. https://doi.org/10.1109/ASRU46091.2019.9003832

Inaguma H, Duh K, Kawahara T, Watanabe S. Multilingual End-To-End Speech Translation. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2019. p. 570-577. 9003832. (2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings). doi: 10.1109/ASRU46091.2019.9003832

@inproceedings{22eb39cdbd8f45b8bc46db3895615cd8,

title = "Multilingual End-To-End Speech Translation",

abstract = "In this paper, we propose a simple yet effective framework for multilingual end-To-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-To-sequence architecture. While multilingual models have shown to be useful for automatic speech recognition (ASR) and machine translation (MT), this is the first time they are applied to the end-To-end ST problem. We show the effectiveness of multilingual end-To-end ST in two scenarios: one-To-many and many-To-many translations with publicly available data. We experimentally confirm that multilingual end-To-end ST models significantly outperform bilingual ones in both scenarios. The generalization of multilingual training is also evaluated in a transfer learning scenario to a very low-resource language pair. All of our codes and the database are publicly available to encourage further research in this emergent multilingual ST topic11Available at https://github.com/espnet/espnet.",

keywords = "Speech translation, attention-based sequence-To-sequence, multilingual end-To-end speech translation, transfer learning",

author = "Hirofumi Inaguma and Kevin Duh and Tatsuya Kawahara and Shinji Watanabe",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 ; Conference date: 15-12-2019 Through 18-12-2019",

year = "2019",

month = dec,

doi = "10.1109/ASRU46091.2019.9003832",

language = "English",

series = "2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "570--577",

booktitle = "2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings",

}

TY - GEN

T1 - Multilingual End-To-End Speech Translation

AU - Inaguma, Hirofumi

AU - Duh, Kevin

AU - Kawahara, Tatsuya

AU - Watanabe, Shinji

PY - 2019/12

Y1 - 2019/12

N2 - In this paper, we propose a simple yet effective framework for multilingual end-To-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-To-sequence architecture. While multilingual models have shown to be useful for automatic speech recognition (ASR) and machine translation (MT), this is the first time they are applied to the end-To-end ST problem. We show the effectiveness of multilingual end-To-end ST in two scenarios: one-To-many and many-To-many translations with publicly available data. We experimentally confirm that multilingual end-To-end ST models significantly outperform bilingual ones in both scenarios. The generalization of multilingual training is also evaluated in a transfer learning scenario to a very low-resource language pair. All of our codes and the database are publicly available to encourage further research in this emergent multilingual ST topic11Available at https://github.com/espnet/espnet.

AB - In this paper, we propose a simple yet effective framework for multilingual end-To-end speech translation (ST), in which speech utterances in source languages are directly translated to the desired target languages with a universal sequence-To-sequence architecture. While multilingual models have shown to be useful for automatic speech recognition (ASR) and machine translation (MT), this is the first time they are applied to the end-To-end ST problem. We show the effectiveness of multilingual end-To-end ST in two scenarios: one-To-many and many-To-many translations with publicly available data. We experimentally confirm that multilingual end-To-end ST models significantly outperform bilingual ones in both scenarios. The generalization of multilingual training is also evaluated in a transfer learning scenario to a very low-resource language pair. All of our codes and the database are publicly available to encourage further research in this emergent multilingual ST topic11Available at https://github.com/espnet/espnet.

KW - Speech translation

KW - attention-based sequence-To-sequence

KW - multilingual end-To-end speech translation

KW - transfer learning

UR - http://www.scopus.com/inward/record.url?scp=85081588714&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85081588714&partnerID=8YFLogxK

U2 - 10.1109/ASRU46091.2019.9003832

DO - 10.1109/ASRU46091.2019.9003832

M3 - Conference contribution

AN - SCOPUS:85081588714

T3 - 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings

SP - 570

EP - 577

BT - 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019

Y2 - 15 December 2019 through 18 December 2019

ER -

Multilingual End-To-End Speech Translation

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this