TY - JOUR
T1 - A conditional entropy minimization criterion for dimensionality reduction and multiple kernel learning
AU - Hino, Hideitsu
AU - Murata, Noboru
PY - 2010/11/1
Y1 - 2010/11/1
N2 - Reducing the dimensionality of high-dimensional data without losing its essential information is an important task in information processing. When class labels of training data are available, Fisher discriminant analysis (FDA) has been widely used. However, the optimality of FDA is guaranteed only in a very restricted ideal circumstance, and it is often observed that FDA does not provide a good classification surface for many real problems. This letter treats the problem of supervised dimensionality reduction from the viewpoint of information theory and proposes a framework of dimensionality reduction based on classconditional entropy minimization. The proposed linear dimensionalityreduction technique is validated both theoretically and experimentally. Then, through kernel Fisher discriminant analysis (KFDA), the multiple kernel learning problem is treated in the proposed framework, and a novel algorithm, which iteratively optimizes the parameters of the classification function and kernel combination coefficients, is proposed. The algorithm is experimentally shown to be comparable to or outperforms KFDA for large-scale benchmark data sets, and comparable to other multiple kernel learning techniques on the yeast protein function annotation task.
AB - Reducing the dimensionality of high-dimensional data without losing its essential information is an important task in information processing. When class labels of training data are available, Fisher discriminant analysis (FDA) has been widely used. However, the optimality of FDA is guaranteed only in a very restricted ideal circumstance, and it is often observed that FDA does not provide a good classification surface for many real problems. This letter treats the problem of supervised dimensionality reduction from the viewpoint of information theory and proposes a framework of dimensionality reduction based on classconditional entropy minimization. The proposed linear dimensionalityreduction technique is validated both theoretically and experimentally. Then, through kernel Fisher discriminant analysis (KFDA), the multiple kernel learning problem is treated in the proposed framework, and a novel algorithm, which iteratively optimizes the parameters of the classification function and kernel combination coefficients, is proposed. The algorithm is experimentally shown to be comparable to or outperforms KFDA for large-scale benchmark data sets, and comparable to other multiple kernel learning techniques on the yeast protein function annotation task.
UR - http://www.scopus.com/inward/record.url?scp=78149311149&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=78149311149&partnerID=8YFLogxK
U2 - 10.1162/NECO_a_00027
DO - 10.1162/NECO_a_00027
M3 - Letter
C2 - 20804381
AN - SCOPUS:78149311149
SN - 0899-7667
VL - 22
SP - 2887
EP - 2923
JO - Neural Computation
JF - Neural Computation
IS - 11
ER -