An entropy estimator based on polynomial regression with poisson error structure

Hideitsu Hino; Shotaro Akaho; Noboru Murata

doi:10.1007/978-3-319-46672-9_2

An entropy estimator based on polynomial regression with poisson error structure

Hideitsu Hino^*, Shotaro Akaho, Noboru Murata

^*Corresponding author for this work

School of Advanced Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

Original language	English
Title of host publication	Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings
Editors	Seiichi Ozawa, Kazushi Ikeda, Derong Liu, Akira Hirose, Kenji Doya, Minho Lee
Publisher	Springer Verlag
Pages	11-19
Number of pages	9
ISBN (Print)	9783319466712
DOIs	https://doi.org/10.1007/978-3-319-46672-9_2
Publication status	Published - 2016
Event	23rd International Conference on Neural Information Processing, ICONIP 2016 - Kyoto, Japan Duration: 2016 Oct 16 → 2016 Oct 21

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9948 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	23rd International Conference on Neural Information Processing, ICONIP 2016
Country/Territory	Japan
City	Kyoto
Period	16/10/16 → 16/10/21

Keywords

Density estimation
Entropy
Poisson error structure
Regression

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-319-46672-9_2

Cite this

Hino, H., Akaho, S., & Murata, N. (2016). An entropy estimator based on polynomial regression with poisson error structure. In S. Ozawa, K. Ikeda, D. Liu, A. Hirose, K. Doya, & M. Lee (Eds.), Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings (pp. 11-19). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9948 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46672-9_2

An entropy estimator based on polynomial regression with poisson error structure. / Hino, Hideitsu; Akaho, Shotaro; Murata, Noboru.
Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. ed. / Seiichi Ozawa; Kazushi Ikeda; Derong Liu; Akira Hirose; Kenji Doya; Minho Lee. Springer Verlag, 2016. p. 11-19 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9948 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hino, H, Akaho, S & Murata, N 2016, An entropy estimator based on polynomial regression with poisson error structure. in S Ozawa, K Ikeda, D Liu, A Hirose, K Doya & M Lee (eds), Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9948 LNCS, Springer Verlag, pp. 11-19, 23rd International Conference on Neural Information Processing, ICONIP 2016, Kyoto, Japan, 16/10/16. https://doi.org/10.1007/978-3-319-46672-9_2

Hino H, Akaho S, Murata N. An entropy estimator based on polynomial regression with poisson error structure. In Ozawa S, Ikeda K, Liu D, Hirose A, Doya K, Lee M, editors, Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. Springer Verlag. 2016. p. 11-19. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-46672-9_2

Hino, Hideitsu ; Akaho, Shotaro ; Murata, Noboru. / An entropy estimator based on polynomial regression with poisson error structure. Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings. editor / Seiichi Ozawa ; Kazushi Ikeda ; Derong Liu ; Akira Hirose ; Kenji Doya ; Minho Lee. Springer Verlag, 2016. pp. 11-19 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a7f097a8164949e89b8c211b0546c851,

title = "An entropy estimator based on polynomial regression with poisson error structure",

abstract = "A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.",

keywords = "Density estimation, Entropy, Poisson error structure, Regression",

author = "Hideitsu Hino and Shotaro Akaho and Noboru Murata",

note = "Funding Information: Part of this work was supported by JSPS KAKENHI No. 25120009, 25120011, and 16K16108.; 23rd International Conference on Neural Information Processing, ICONIP 2016 ; Conference date: 16-10-2016 Through 21-10-2016",

year = "2016",

doi = "10.1007/978-3-319-46672-9_2",

language = "English",

isbn = "9783319466712",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "11--19",

editor = "Seiichi Ozawa and Kazushi Ikeda and Derong Liu and Akira Hirose and Kenji Doya and Minho Lee",

booktitle = "Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings",

}

TY - GEN

T1 - An entropy estimator based on polynomial regression with poisson error structure

AU - Hino, Hideitsu

AU - Akaho, Shotaro

AU - Murata, Noboru

N1 - Funding Information: Part of this work was supported by JSPS KAKENHI No. 25120009, 25120011, and 16K16108.

PY - 2016

Y1 - 2016

N2 - A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

AB - A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

KW - Density estimation

KW - Entropy

KW - Poisson error structure

KW - Regression

UR - http://www.scopus.com/inward/record.url?scp=84992533118&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84992533118&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46672-9_2

DO - 10.1007/978-3-319-46672-9_2

M3 - Conference contribution

AN - SCOPUS:84992533118

SN - 9783319466712

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 11

EP - 19

BT - Neural Information Processing - 23rd International Conference, ICONIP 2016, Proceedings

A2 - Ozawa, Seiichi

A2 - Ikeda, Kazushi

A2 - Liu, Derong

A2 - Hirose, Akira

A2 - Doya, Kenji

A2 - Lee, Minho

PB - Springer Verlag

T2 - 23rd International Conference on Neural Information Processing, ICONIP 2016

Y2 - 16 October 2016 through 21 October 2016

ER -

An entropy estimator based on polynomial regression with poisson error structure

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this