Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining

Masayuki Goto; Takashi Ishida; Makoto Suzuki; Shigeichi Hirasawa

doi:10.1109/ISITA.2008.4895453

Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining

Masayuki Goto^*, Takashi Ishida, Makoto Suzuki, Shigeichi Hirasawa

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

This paper discusses the document classification problems in text mining from the viewpoint of asymptotic statistical analysis. In the problem of text mining, the several heuristics are applied to practical analysis because of its experimental effectiveness in many case studies. The theoretical explanation about the performance of text mining techniques is required and such thinking will give us very clear idea. In this paper, the performances of distance measures used to classify the documents are analyzed from the new viewpoint of asymptotic analysis. We also discuss the asymptotic performance of IDF measure used in the information retrieval field.

Original language	English
Title of host publication	2008 International Symposium on Information Theory and its Applications, ISITA2008
DOIs	https://doi.org/10.1109/ISITA.2008.4895453
Publication status	Published - 2008
Externally published	Yes
Event	2008 International Symposium on Information Theory and its Applications, ISITA2008 - Auckland, New Zealand Duration: 2008 Dec 7 → 2008 Dec 10

Publication series

Name	2008 International Symposium on Information Theory and its Applications, ISITA2008

Conference

Conference	2008 International Symposium on Information Theory and its Applications, ISITA2008
Country/Territory	New Zealand
City	Auckland
Period	08/12/7 → 08/12/10

ASJC Scopus subject areas

Computer Science(all)

Access to Document

10.1109/ISITA.2008.4895453

Cite this

Goto, M., Ishida, T., Suzuki, M., & Hirasawa, S. (2008). Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining. In 2008 International Symposium on Information Theory and its Applications, ISITA2008 Article 4895453 (2008 International Symposium on Information Theory and its Applications, ISITA2008). https://doi.org/10.1109/ISITA.2008.4895453

Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining. / Goto, Masayuki; Ishida, Takashi; Suzuki, Makoto et al.
2008 International Symposium on Information Theory and its Applications, ISITA2008. 2008. 4895453 (2008 International Symposium on Information Theory and its Applications, ISITA2008).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Goto, M, Ishida, T, Suzuki, M & Hirasawa, S 2008, Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining. in 2008 International Symposium on Information Theory and its Applications, ISITA2008., 4895453, 2008 International Symposium on Information Theory and its Applications, ISITA2008, 2008 International Symposium on Information Theory and its Applications, ISITA2008, Auckland, New Zealand, 08/12/7. https://doi.org/10.1109/ISITA.2008.4895453

@inproceedings{2354fc4170fb4a06a0a6f7897212712f,

title = "Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining",

abstract = "This paper discusses the document classification problems in text mining from the viewpoint of asymptotic statistical analysis. In the problem of text mining, the several heuristics are applied to practical analysis because of its experimental effectiveness in many case studies. The theoretical explanation about the performance of text mining techniques is required and such thinking will give us very clear idea. In this paper, the performances of distance measures used to classify the documents are analyzed from the new viewpoint of asymptotic analysis. We also discuss the asymptotic performance of IDF measure used in the information retrieval field.",

author = "Masayuki Goto and Takashi Ishida and Makoto Suzuki and Shigeichi Hirasawa",

year = "2008",

doi = "10.1109/ISITA.2008.4895453",

language = "English",

isbn = "9781424420698",

series = "2008 International Symposium on Information Theory and its Applications, ISITA2008",

booktitle = "2008 International Symposium on Information Theory and its Applications, ISITA2008",

note = "2008 International Symposium on Information Theory and its Applications, ISITA2008 ; Conference date: 07-12-2008 Through 10-12-2008",

}

TY - GEN

T1 - Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining

AU - Goto, Masayuki

AU - Ishida, Takashi

AU - Suzuki, Makoto

AU - Hirasawa, Shigeichi

PY - 2008

Y1 - 2008

N2 - This paper discusses the document classification problems in text mining from the viewpoint of asymptotic statistical analysis. In the problem of text mining, the several heuristics are applied to practical analysis because of its experimental effectiveness in many case studies. The theoretical explanation about the performance of text mining techniques is required and such thinking will give us very clear idea. In this paper, the performances of distance measures used to classify the documents are analyzed from the new viewpoint of asymptotic analysis. We also discuss the asymptotic performance of IDF measure used in the information retrieval field.

AB - This paper discusses the document classification problems in text mining from the viewpoint of asymptotic statistical analysis. In the problem of text mining, the several heuristics are applied to practical analysis because of its experimental effectiveness in many case studies. The theoretical explanation about the performance of text mining techniques is required and such thinking will give us very clear idea. In this paper, the performances of distance measures used to classify the documents are analyzed from the new viewpoint of asymptotic analysis. We also discuss the asymptotic performance of IDF measure used in the information retrieval field.

UR - http://www.scopus.com/inward/record.url?scp=77951132642&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77951132642&partnerID=8YFLogxK

U2 - 10.1109/ISITA.2008.4895453

DO - 10.1109/ISITA.2008.4895453

M3 - Conference contribution

AN - SCOPUS:77951132642

SN - 9781424420698

T3 - 2008 International Symposium on Information Theory and its Applications, ISITA2008

BT - 2008 International Symposium on Information Theory and its Applications, ISITA2008

T2 - 2008 International Symposium on Information Theory and its Applications, ISITA2008

Y2 - 7 December 2008 through 10 December 2008

ER -

Asymptotic evaluation of distance measure on high dimensional vector spaces in text mining

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this