Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR

Felix Weninger; Hakan Erdogan; Shinji Watanabe; Emmanuel Vincent; Jonathan Le Roux; John R. Hershey; Björn Schuller

doi:10.1007/978-3-319-22482-4_11

Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR

Felix Weninger^*, Hakan Erdogan, Shinji Watanabe, Emmanuel Vincent, Jonathan Le Roux, John R. Hershey, Björn Schuller

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

414 Citations (Scopus)

Abstract

We evaluate some recent developments in recurrent neural network (RNN) based speech enhancement in the light of noise-robust automatic speech recognition (ASR). The proposed framework is based on Long Short-Term Memory (LSTM) RNNs which are discriminatively trained according to an optimal speech reconstruction objective. We demonstrate that LSTM speech enhancement, even when used ‘naïvely’ as front-end processing, delivers competitive results on the CHiME-2 speech recognition task. Furthermore, simple, feature-level fusion based extensions to the framework are proposed to improve the integration with the ASR back-end. These yield a best result of 13.76% average word error rate, which is, to our knowledge, the best score to date.

Original language	English
Title of host publication	Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings
Editors	Zbynĕk Koldovský, Emmanuel Vincent, Arie Yeredor, Petr Tichavský
Publisher	Springer Verlag
Pages	91-99
Number of pages	9
ISBN (Print)	9783319224817
DOIs	https://doi.org/10.1007/978-3-319-22482-4_11
Publication status	Published - 2015
Externally published	Yes
Event	12th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2015 - Liberec, Czech Republic Duration: 2015 Aug 25 → 2015 Aug 28

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9237
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	12th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2015
Country/Territory	Czech Republic
City	Liberec
Period	15/8/25 → 15/8/28

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-319-22482-4_11

Cite this

Weninger, F., Erdogan, H., Watanabe, S., Vincent, E., Le Roux, J., Hershey, J. R., & Schuller, B. (2015). Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR. In Z. Koldovský, E. Vincent, A. Yeredor, & P. Tichavský (Eds.), Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings (pp. 91-99). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9237). Springer Verlag. https://doi.org/10.1007/978-3-319-22482-4_11

Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR. / Weninger, Felix; Erdogan, Hakan; Watanabe, Shinji et al.
Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings. ed. / Zbynĕk Koldovský; Emmanuel Vincent; Arie Yeredor; Petr Tichavský. Springer Verlag, 2015. p. 91-99 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9237).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Weninger, F, Erdogan, H, Watanabe, S, Vincent, E, Le Roux, J, Hershey, JR & Schuller, B 2015, Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR. in Z Koldovský, E Vincent, A Yeredor & P Tichavský (eds), Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9237, Springer Verlag, pp. 91-99, 12th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2015, Liberec, Czech Republic, 15/8/25. https://doi.org/10.1007/978-3-319-22482-4_11

Weninger F, Erdogan H, Watanabe S, Vincent E, Le Roux J, Hershey JR et al. Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR. In Koldovský Z, Vincent E, Yeredor A, Tichavský P, editors, Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings. Springer Verlag. 2015. p. 91-99. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-22482-4_11

Weninger, Felix ; Erdogan, Hakan ; Watanabe, Shinji et al. / Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR. Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings. editor / Zbynĕk Koldovský ; Emmanuel Vincent ; Arie Yeredor ; Petr Tichavský. Springer Verlag, 2015. pp. 91-99 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{7c4685bb91804a2a93e86f28c8a25199,

title = "Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR",

abstract = "We evaluate some recent developments in recurrent neural network (RNN) based speech enhancement in the light of noise-robust automatic speech recognition (ASR). The proposed framework is based on Long Short-Term Memory (LSTM) RNNs which are discriminatively trained according to an optimal speech reconstruction objective. We demonstrate that LSTM speech enhancement, even when used {\textquoteleft}na{\"i}vely{\textquoteright} as front-end processing, delivers competitive results on the CHiME-2 speech recognition task. Furthermore, simple, feature-level fusion based extensions to the framework are proposed to improve the integration with the ASR back-end. These yield a best result of 13.76% average word error rate, which is, to our knowledge, the best score to date.",

author = "Felix Weninger and Hakan Erdogan and Shinji Watanabe and Emmanuel Vincent and {Le Roux}, Jonathan and Hershey, {John R.} and Bj{\"o}rn Schuller",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2015.; 12th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2015 ; Conference date: 25-08-2015 Through 28-08-2015",

year = "2015",

doi = "10.1007/978-3-319-22482-4_11",

language = "English",

isbn = "9783319224817",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "91--99",

editor = "Zbynĕk Koldovsk{\'y} and Emmanuel Vincent and Arie Yeredor and Petr Tichavsk{\'y}",

booktitle = "Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings",

}

TY - GEN

T1 - Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR

AU - Weninger, Felix

AU - Erdogan, Hakan

AU - Watanabe, Shinji

AU - Vincent, Emmanuel

AU - Le Roux, Jonathan

AU - Hershey, John R.

AU - Schuller, Björn

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2015.

PY - 2015

Y1 - 2015

N2 - We evaluate some recent developments in recurrent neural network (RNN) based speech enhancement in the light of noise-robust automatic speech recognition (ASR). The proposed framework is based on Long Short-Term Memory (LSTM) RNNs which are discriminatively trained according to an optimal speech reconstruction objective. We demonstrate that LSTM speech enhancement, even when used ‘naïvely’ as front-end processing, delivers competitive results on the CHiME-2 speech recognition task. Furthermore, simple, feature-level fusion based extensions to the framework are proposed to improve the integration with the ASR back-end. These yield a best result of 13.76% average word error rate, which is, to our knowledge, the best score to date.

AB - We evaluate some recent developments in recurrent neural network (RNN) based speech enhancement in the light of noise-robust automatic speech recognition (ASR). The proposed framework is based on Long Short-Term Memory (LSTM) RNNs which are discriminatively trained according to an optimal speech reconstruction objective. We demonstrate that LSTM speech enhancement, even when used ‘naïvely’ as front-end processing, delivers competitive results on the CHiME-2 speech recognition task. Furthermore, simple, feature-level fusion based extensions to the framework are proposed to improve the integration with the ASR back-end. These yield a best result of 13.76% average word error rate, which is, to our knowledge, the best score to date.

UR - http://www.scopus.com/inward/record.url?scp=84944675581&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944675581&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-22482-4_11

DO - 10.1007/978-3-319-22482-4_11

M3 - Conference contribution

AN - SCOPUS:84944675581

SN - 9783319224817

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 91

EP - 99

BT - Latent Variable Analysis and Signal Separation - 12th International Conference, LVA/ICA 2015, Proceedings

A2 - Koldovský, Zbynĕk

A2 - Vincent, Emmanuel

A2 - Yeredor, Arie

A2 - Tichavský, Petr

PB - Springer Verlag

T2 - 12th International Conference on Latent Variable Analysis and Signal Separation, LVA/ICA 2015

Y2 - 25 August 2015 through 28 August 2015

ER -

Speech enhancement with LSTM recurrent neural networks and its application to noise-robust ASR

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this