TY - GEN
T1 - Statistical analysis of regularization constant from bayes, MDL and NIC Points of view
AU - Amari, Shun Ichi
AU - Murata, Noboru
PY - 1997/1/1
Y1 - 1997/1/1
N2 - In order to avoid overfitting in neural learning, a regularization term is added to the loss function to be minimized. It is naturMly derived from the Bayesian standpoint. The present paper studies how to determine the regularization constant from the points of view of the empirical Bayes approach, the maximum description length (MDL) approach, and the network information criterion (NIC) approach. The asymptotic statistical analysis is given to elucidate their differences. These approaches are tightly connected with the method of model selection. The superiority of the NIC is shown from this analysis.
AB - In order to avoid overfitting in neural learning, a regularization term is added to the loss function to be minimized. It is naturMly derived from the Bayesian standpoint. The present paper studies how to determine the regularization constant from the points of view of the empirical Bayes approach, the maximum description length (MDL) approach, and the network information criterion (NIC) approach. The asymptotic statistical analysis is given to elucidate their differences. These approaches are tightly connected with the method of model selection. The superiority of the NIC is shown from this analysis.
UR - http://www.scopus.com/inward/record.url?scp=21744450484&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=21744450484&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:21744450484
SN - 3540630473
SN - 9783540630470
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 284
EP - 293
BT - Biological and Artificial Computation
PB - Springer Verlag
T2 - 4th International Work-Conference on Artificial and Natural Neural Networks, IWANN 1997
Y2 - 4 June 1997 through 6 June 1997
ER -