Network Information Criterion—Determining the Number of Hidden Units for an Artificial Neural Network Model

Noboru Murata; Shuji Yoshizawa; Shun Ichi Amari

doi:10.1109/72.329683

Network Information Criterion—Determining the Number of Hidden Units for an Artificial Neural Network Model

Noboru Murata, Shuji Yoshizawa, Shun Ichi Amari

Research output: Contribution to journal › Article › peer-review

459 Citations (Scopus)

Abstract

The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike’s information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

Original language	English
Pages (from-to)	865-872
Number of pages	8
Journal	IEEE Transactions on Neural Networks
Volume	5
Issue number	6
DOIs	https://doi.org/10.1109/72.329683
Publication status	Published - 1994 Nov
Externally published	Yes

ASJC Scopus subject areas

Software
Computer Science Applications
Computer Networks and Communications
Artificial Intelligence

Access to Document

10.1109/72.329683

Cite this

@article{091edff2e73846739c93e5ac2847bab0,

title = "Network Information Criterion—Determining the Number of Hidden Units for an Artificial Neural Network Model",

abstract = "The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike{\textquoteright}s information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.",

author = "Noboru Murata and Shuji Yoshizawa and Amari, {Shun Ichi}",

note = "Funding Information: Manuscript received June 29, 1992; revised December 2, 1993. This work was supported in part by grant-in-aid 03251102 and 03234109 from the Ministry of Education of Japan. The authors are with the Department of Mathematical Engineering and Information Physics, Faculty of Engineering, University of Tokyo, Bunkyo-ku, Tokyo 113, Japan. IEEE Log Number 9400108.",

year = "1994",

month = nov,

doi = "10.1109/72.329683",

language = "English",

volume = "5",

pages = "865--872",

journal = "IEEE Transactions on Neural Networks",

issn = "1045-9227",

publisher = "IEEE Computational Intelligence Society",

number = "6",

}

TY - JOUR

T1 - Network Information Criterion—Determining the Number of Hidden Units for an Artificial Neural Network Model

AU - Murata, Noboru

AU - Yoshizawa, Shuji

AU - Amari, Shun Ichi

N1 - Funding Information: Manuscript received June 29, 1992; revised December 2, 1993. This work was supported in part by grant-in-aid 03251102 and 03234109 from the Ministry of Education of Japan. The authors are with the Department of Mathematical Engineering and Information Physics, Faculty of Engineering, University of Tokyo, Bunkyo-ku, Tokyo 113, Japan. IEEE Log Number 9400108.

PY - 1994/11

Y1 - 1994/11

N2 - The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike’s information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

AB - The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike’s information criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of a network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

UR - http://www.scopus.com/inward/record.url?scp=0028544395&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0028544395&partnerID=8YFLogxK

U2 - 10.1109/72.329683

DO - 10.1109/72.329683

M3 - Article

AN - SCOPUS:0028544395

SN - 1045-9227

VL - 5

SP - 865

EP - 872

JO - IEEE Transactions on Neural Networks

JF - IEEE Transactions on Neural Networks

IS - 6

ER -

Network Information Criterion—Determining the Number of Hidden Units for an Artificial Neural Network Model

Abstract

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this