End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection

Yuki Takashima; Yusuke Fujita; Shinji Watanabe; Shota Horiguchi; Paola Garcia; Kenji Nagamatsu

doi:10.1109/SLT48900.2021.9383555

End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection

Yuki Takashima, Yusuke Fujita, Shinji Watanabe, Shota Horiguchi, Paola Garcia, Kenji Nagamatsu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

16 Citations (Scopus)

Abstract

In this paper, we present a conditional multitask learning method for end-to-end neural speaker diarization (EEND). The EEND system has shown promising performance compared with traditional clustering-based methods, especially in the case of overlapping speech. In this paper, to further improve the performance of the EEND system, we propose a novel multitask learning framework that solves speaker diarization and a desired subtask while explicitly considering the task dependency. We optimize speaker diarization conditioned on speech activity and overlap detection that are subtasks of speaker diarization, based on the probabilistic chain rule. Experimental results show that our proposed method can leverage a subtask to effectively model speaker diarization, and outperforms conventional EEND systems in terms of diarization error rate.

Original language	English
Title of host publication	2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	849-856
Number of pages	8
ISBN (Electronic)	9781728170664
DOIs	https://doi.org/10.1109/SLT48900.2021.9383555
Publication status	Published - 2021 Jan 19
Externally published	Yes
Event	2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Virtual, Shenzhen, China Duration: 2021 Jan 19 → 2021 Jan 22

Publication series

Name	2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings

Conference

Conference	2021 IEEE Spoken Language Technology Workshop, SLT 2021
Country/Territory	China
City	Virtual, Shenzhen
Period	21/1/19 → 21/1/22

Keywords

chain rule
end-to-end
multitask learning
neural network
speaker diarization

ASJC Scopus subject areas

Linguistics and Language
Language and Linguistics
Artificial Intelligence
Computer Science Applications
Computer Vision and Pattern Recognition
Hardware and Architecture

Access to Document

10.1109/SLT48900.2021.9383555

Cite this

Takashima, Y., Fujita, Y., Watanabe, S., Horiguchi, S., Garcia, P., & Nagamatsu, K. (2021). End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection. In 2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings (pp. 849-856). Article 9383555 (2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SLT48900.2021.9383555

End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection. / Takashima, Yuki; Fujita, Yusuke; Watanabe, Shinji et al.
2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2021. p. 849-856 9383555 (2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Takashima, Y, Fujita, Y, Watanabe, S, Horiguchi, S, Garcia, P & Nagamatsu, K 2021, End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection. in 2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings., 9383555, 2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 849-856, 2021 IEEE Spoken Language Technology Workshop, SLT 2021, Virtual, Shenzhen, China, 21/1/19. https://doi.org/10.1109/SLT48900.2021.9383555

Takashima Y, Fujita Y, Watanabe S, Horiguchi S, Garcia P, Nagamatsu K. End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection. In 2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2021. p. 849-856. 9383555. (2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings). doi: 10.1109/SLT48900.2021.9383555

@inproceedings{d72c43643c774ebf991075a3ec85d565,

title = "End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection",

abstract = "In this paper, we present a conditional multitask learning method for end-to-end neural speaker diarization (EEND). The EEND system has shown promising performance compared with traditional clustering-based methods, especially in the case of overlapping speech. In this paper, to further improve the performance of the EEND system, we propose a novel multitask learning framework that solves speaker diarization and a desired subtask while explicitly considering the task dependency. We optimize speaker diarization conditioned on speech activity and overlap detection that are subtasks of speaker diarization, based on the probabilistic chain rule. Experimental results show that our proposed method can leverage a subtask to effectively model speaker diarization, and outperforms conventional EEND systems in terms of diarization error rate.",

keywords = "chain rule, end-to-end, multitask learning, neural network, speaker diarization",

author = "Yuki Takashima and Yusuke Fujita and Shinji Watanabe and Shota Horiguchi and Paola Garcia and Kenji Nagamatsu",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE Spoken Language Technology Workshop, SLT 2021 ; Conference date: 19-01-2021 Through 22-01-2021",

year = "2021",

month = jan,

day = "19",

doi = "10.1109/SLT48900.2021.9383555",

language = "English",

series = "2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "849--856",

booktitle = "2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings",

}

TY - GEN

T1 - End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection

AU - Takashima, Yuki

AU - Fujita, Yusuke

AU - Watanabe, Shinji

AU - Horiguchi, Shota

AU - Garcia, Paola

AU - Nagamatsu, Kenji

PY - 2021/1/19

Y1 - 2021/1/19

N2 - In this paper, we present a conditional multitask learning method for end-to-end neural speaker diarization (EEND). The EEND system has shown promising performance compared with traditional clustering-based methods, especially in the case of overlapping speech. In this paper, to further improve the performance of the EEND system, we propose a novel multitask learning framework that solves speaker diarization and a desired subtask while explicitly considering the task dependency. We optimize speaker diarization conditioned on speech activity and overlap detection that are subtasks of speaker diarization, based on the probabilistic chain rule. Experimental results show that our proposed method can leverage a subtask to effectively model speaker diarization, and outperforms conventional EEND systems in terms of diarization error rate.

AB - In this paper, we present a conditional multitask learning method for end-to-end neural speaker diarization (EEND). The EEND system has shown promising performance compared with traditional clustering-based methods, especially in the case of overlapping speech. In this paper, to further improve the performance of the EEND system, we propose a novel multitask learning framework that solves speaker diarization and a desired subtask while explicitly considering the task dependency. We optimize speaker diarization conditioned on speech activity and overlap detection that are subtasks of speaker diarization, based on the probabilistic chain rule. Experimental results show that our proposed method can leverage a subtask to effectively model speaker diarization, and outperforms conventional EEND systems in terms of diarization error rate.

KW - chain rule

KW - end-to-end

KW - multitask learning

KW - neural network

KW - speaker diarization

UR - http://www.scopus.com/inward/record.url?scp=85103974334&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85103974334&partnerID=8YFLogxK

U2 - 10.1109/SLT48900.2021.9383555

DO - 10.1109/SLT48900.2021.9383555

M3 - Conference contribution

AN - SCOPUS:85103974334

T3 - 2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings

SP - 849

EP - 856

BT - 2021 IEEE Spoken Language Technology Workshop, SLT 2021 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2021 IEEE Spoken Language Technology Workshop, SLT 2021

Y2 - 19 January 2021 through 22 January 2021

ER -

End-to-End Speaker Diarization Conditioned on Speech Activity and Overlap Detection

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this