System evaluation of ternary error-correcting output codes for multiclass classification problems

To solve multiple classification problems with M (geq 3) categories, many studies have been devoted using N (geq lceillog-{2}Mrceil) binary ({0, 1}) classifiers, where these systems are known as binary Error-Correcting Output Codes (binary ECOC). As an extended version of the binary ECOC, the ternary ({0,∗1}) ECOC have also been discussed, where ternary classifiers classify data into positive examples when the element is 1, into negative examples when the element is 0, and no classification when the element is. In this paper, we discuss the ternary ECOC system from the view point of the system evaluation model based on rate-distortion function. First, we discuss a table of M code words with length N which is given by a ternary matrix W of M rows and N columns. Next, by leveraging the benchmark data for multiclass document classification which is widely used in Japan, the relationships between the probability of classification error P_e and the number of the ternary classifiers N for a given M are experimentally investigated. In addition, by assuming the M-dimensional Normal distribution for a classification data model, the relationship between P_e and N for a given M is also examined. Finally, we show by the system evaluation model that the ternary ECOC systems have desirable properties such as 'Flexible', 'Elastic', and 'Effective Elastic', when M becomes large.