Juman++: A morphological analysis toolkit for scriptio continua

Arseny Tolmachev; Daisuke Kawahara; Sadao Kurohashi

doi:10.18653/v1/d18-2010

Juman++: A morphological analysis toolkit for scriptio continua

Arseny Tolmachev, Daisuke Kawahara, Sadao Kurohashi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

34 Citations (Scopus)

Abstract

We present a three-part toolkit for developing morphological analyzers for languages without natural word boundaries. The first part is a lattice-based morphological analysis library that uses a combination of linear and recurrent neural net language models for analysis. The other parts are a tool for exposing problems in the trained model and a partial annotation tool. Our morphological analyzer for Japanese achieves new SOTA on Jumandic-based corpora while being 250 times faster than the previous one. We also perform a small experiment and quantitive analysis of using our toolkit.

Original language	English
Title of host publication	EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing
Subtitle of host publication	System Demonstrations, Proceedings
Publisher	Association for Computational Linguistics (ACL)
Pages	54-59
Number of pages	6
ISBN (Electronic)	9781948087858
DOIs	https://doi.org/10.18653/v1/d18-2010
Publication status	Published - 2018
Externally published	Yes
Event	2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2018 - Brussels, Belgium Duration: 2018 Oct 31 → 2018 Nov 4

Publication series

Name	EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings

Conference

Conference	2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2018
Country/Territory	Belgium
City	Brussels
Period	18/10/31 → 18/11/4

ASJC Scopus subject areas

Information Systems
Computational Theory and Mathematics
Computer Science Applications

Access to Document

10.18653/v1/d18-2010

Cite this

Tolmachev, A., Kawahara, D., & Kurohashi, S. (2018). Juman++: A morphological analysis toolkit for scriptio continua. In EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings (pp. 54-59). (EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/d18-2010

Juman++: A morphological analysis toolkit for scriptio continua. / Tolmachev, Arseny; Kawahara, Daisuke; Kurohashi, Sadao.
EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings. Association for Computational Linguistics (ACL), 2018. p. 54-59 (EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Tolmachev, A, Kawahara, D & Kurohashi, S 2018, Juman++: A morphological analysis toolkit for scriptio continua. in EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings. EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings, Association for Computational Linguistics (ACL), pp. 54-59, 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2018, Brussels, Belgium, 18/10/31. https://doi.org/10.18653/v1/d18-2010

Tolmachev A, Kawahara D, Kurohashi S. Juman++: A morphological analysis toolkit for scriptio continua. In EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings. Association for Computational Linguistics (ACL). 2018. p. 54-59. (EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings). doi: 10.18653/v1/d18-2010

Tolmachev, Arseny ; Kawahara, Daisuke ; Kurohashi, Sadao. / Juman++ : A morphological analysis toolkit for scriptio continua. EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings. Association for Computational Linguistics (ACL), 2018. pp. 54-59 (EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings).

@inproceedings{977230331500450285f470eba6686960,

title = "Juman++: A morphological analysis toolkit for scriptio continua",

abstract = "We present a three-part toolkit for developing morphological analyzers for languages without natural word boundaries. The first part is a lattice-based morphological analysis library that uses a combination of linear and recurrent neural net language models for analysis. The other parts are a tool for exposing problems in the trained model and a partial annotation tool. Our morphological analyzer for Japanese achieves new SOTA on Jumandic-based corpora while being 250 times faster than the previous one. We also perform a small experiment and quantitive analysis of using our toolkit.",

author = "Arseny Tolmachev and Daisuke Kawahara and Sadao Kurohashi",

note = "Publisher Copyright: {\textcopyright} 2018 Association for Computational Linguistics.; 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2018 ; Conference date: 31-10-2018 Through 04-11-2018",

year = "2018",

doi = "10.18653/v1/d18-2010",

language = "English",

series = "EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings",

publisher = "Association for Computational Linguistics (ACL)",

pages = "54--59",

booktitle = "EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing",

}

TY - GEN

T1 - Juman++

T2 - 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP 2018

AU - Tolmachev, Arseny

AU - Kawahara, Daisuke

AU - Kurohashi, Sadao

PY - 2018

Y1 - 2018

N2 - We present a three-part toolkit for developing morphological analyzers for languages without natural word boundaries. The first part is a lattice-based morphological analysis library that uses a combination of linear and recurrent neural net language models for analysis. The other parts are a tool for exposing problems in the trained model and a partial annotation tool. Our morphological analyzer for Japanese achieves new SOTA on Jumandic-based corpora while being 250 times faster than the previous one. We also perform a small experiment and quantitive analysis of using our toolkit.

AB - We present a three-part toolkit for developing morphological analyzers for languages without natural word boundaries. The first part is a lattice-based morphological analysis library that uses a combination of linear and recurrent neural net language models for analysis. The other parts are a tool for exposing problems in the trained model and a partial annotation tool. Our morphological analyzer for Japanese achieves new SOTA on Jumandic-based corpora while being 250 times faster than the previous one. We also perform a small experiment and quantitive analysis of using our toolkit.

UR - http://www.scopus.com/inward/record.url?scp=85072971404&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072971404&partnerID=8YFLogxK

U2 - 10.18653/v1/d18-2010

DO - 10.18653/v1/d18-2010

M3 - Conference contribution

AN - SCOPUS:85072971404

T3 - EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings

SP - 54

EP - 59

BT - EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing

PB - Association for Computational Linguistics (ACL)

Y2 - 31 October 2018 through 4 November 2018

ER -

Juman++: A morphological analysis toolkit for scriptio continua

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this