A computationally efficient information estimator for weighted data

Hideitsu Hino; Noboru Murata

doi:10.1007/978-3-642-21738-8_39

A computationally efficient information estimator for weighted data

Hideitsu Hino^*, Noboru Murata

^*Corresponding author for this work

School of Advanced Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

The Shannon information content is a fundamental quantity and it is of great importance to estimate it from observed dataset in the field of statistics, information theory, and machine learning. In this study, an estimator for the information content using a given set of weighted data is proposed. The empirical data distribution varies depending on the weight. The notable features of the proposed estimator are its computational efficiency and its ability to deal with weighted data. The proposed estimator is extended in order to estimate cross entropy, entropy and KL divergence with weighted data. Then, the estimators are applied to classification with one-class samples, and distribution preserving data compression problems.

Original language	English
Title of host publication	Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings
Pages	301-308
Number of pages	8
Edition	PART 2
DOIs	https://doi.org/10.1007/978-3-642-21738-8_39
Publication status	Published - 2011
Event	21st International Conference on Artificial Neural Networks, ICANN 2011 - Espoo, Finland Duration: 2011 Jun 14 → 2011 Jun 17

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 2
Volume	6792 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	21st International Conference on Artificial Neural Networks, ICANN 2011
Country/Territory	Finland
City	Espoo
Period	11/6/14 → 11/6/17

Keywords

Information
entropy
non-parametric
quantile

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-642-21738-8_39

Cite this

Hino, H., & Murata, N. (2011). A computationally efficient information estimator for weighted data. In Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings (PART 2 ed., pp. 301-308). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6792 LNCS, No. PART 2). https://doi.org/10.1007/978-3-642-21738-8_39

A computationally efficient information estimator for weighted data. / Hino, Hideitsu; Murata, Noboru.
Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings. PART 2. ed. 2011. p. 301-308 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6792 LNCS, No. PART 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hino, H & Murata, N 2011, A computationally efficient information estimator for weighted data. in Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings. PART 2 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 2, vol. 6792 LNCS, pp. 301-308, 21st International Conference on Artificial Neural Networks, ICANN 2011, Espoo, Finland, 11/6/14. https://doi.org/10.1007/978-3-642-21738-8_39

Hino H, Murata N. A computationally efficient information estimator for weighted data. In Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings. PART 2 ed. 2011. p. 301-308. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2). doi: 10.1007/978-3-642-21738-8_39

Hino, Hideitsu ; Murata, Noboru. / A computationally efficient information estimator for weighted data. Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings. PART 2. ed. 2011. pp. 301-308 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2).

@inproceedings{87dd84bff8604eb2b689367c9c310ef3,

title = "A computationally efficient information estimator for weighted data",

abstract = "The Shannon information content is a fundamental quantity and it is of great importance to estimate it from observed dataset in the field of statistics, information theory, and machine learning. In this study, an estimator for the information content using a given set of weighted data is proposed. The empirical data distribution varies depending on the weight. The notable features of the proposed estimator are its computational efficiency and its ability to deal with weighted data. The proposed estimator is extended in order to estimate cross entropy, entropy and KL divergence with weighted data. Then, the estimators are applied to classification with one-class samples, and distribution preserving data compression problems.",

keywords = "Information, entropy, non-parametric, quantile",

author = "Hideitsu Hino and Noboru Murata",

year = "2011",

doi = "10.1007/978-3-642-21738-8_39",

language = "English",

isbn = "9783642217371",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

number = "PART 2",

pages = "301--308",

booktitle = "Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings",

edition = "PART 2",

note = "21st International Conference on Artificial Neural Networks, ICANN 2011 ; Conference date: 14-06-2011 Through 17-06-2011",

}

TY - GEN

T1 - A computationally efficient information estimator for weighted data

AU - Hino, Hideitsu

AU - Murata, Noboru

PY - 2011

Y1 - 2011

N2 - The Shannon information content is a fundamental quantity and it is of great importance to estimate it from observed dataset in the field of statistics, information theory, and machine learning. In this study, an estimator for the information content using a given set of weighted data is proposed. The empirical data distribution varies depending on the weight. The notable features of the proposed estimator are its computational efficiency and its ability to deal with weighted data. The proposed estimator is extended in order to estimate cross entropy, entropy and KL divergence with weighted data. Then, the estimators are applied to classification with one-class samples, and distribution preserving data compression problems.

AB - The Shannon information content is a fundamental quantity and it is of great importance to estimate it from observed dataset in the field of statistics, information theory, and machine learning. In this study, an estimator for the information content using a given set of weighted data is proposed. The empirical data distribution varies depending on the weight. The notable features of the proposed estimator are its computational efficiency and its ability to deal with weighted data. The proposed estimator is extended in order to estimate cross entropy, entropy and KL divergence with weighted data. Then, the estimators are applied to classification with one-class samples, and distribution preserving data compression problems.

KW - Information

KW - entropy

KW - non-parametric

KW - quantile

UR - http://www.scopus.com/inward/record.url?scp=79959347694&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79959347694&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-21738-8_39

DO - 10.1007/978-3-642-21738-8_39

M3 - Conference contribution

AN - SCOPUS:79959347694

SN - 9783642217371

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 301

EP - 308

BT - Artificial Neural Networks and Machine Learning, ICANN 2011 - 21st International Conference on Artificial Neural Networks, Proceedings

T2 - 21st International Conference on Artificial Neural Networks, ICANN 2011

Y2 - 14 June 2011 through 17 June 2011

ER -

A computationally efficient information estimator for weighted data

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this