A study on difference of codelengths between codes based on MDL principle and bayes codes for given prior distributions

The principle of the Minimum Description Length (MDL) proposed by J. Rissanen provides a type of structure for the model estimation based on probabilistic model selection allowing minimization of the codelength. On the other hand, the use of Bayes codes makes it possible to find a coding function from a mix of probabilistic models without specifying any concrete model. It has been pointed out that codes based on the MDL principle (MDL codes) are closely related to Bayes theory because in the definition of the description length of the probabilistic model, an unknown prior distribution is assumed. In this paper, we apply asymptotic analysis to the codelength difference between the MDL codes and Bayes codes, including cases of different prior distributions. The results of the analysis clearly show that in the case of discrete model families, codes having a high prior distribution in true models (that is, the models for which an advantageous prior distribution is assumed) are favorable, but in the case of parametric model families, Bayes codes have shorter codelength than the MDL codes even in the cases of advantageous prior distribution assumed for the MDL codes.