An information theoretic perspective of the sparse coding

Hideitsu Hino*, Noboru Murata

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)


The sparse coding method is formulated as an information theoretic optimization problem. The rate distortion theory leads to an objective functional which can be interpreted as an information theoretic formulation of the sparse coding. Viewing as an entropy minimization problem, the rate distortion theory and consequently the sparse coding are extended to discriminative variants. As a concrete example of this information theoretic sparse coding, a discriminative non-linear sparse coding algorithm with neural networks is proposed. Experimental results of gender classification by face images show that the discriminative sparse coding is more robust to noise, compared to the conventional method which directly uses images as inputs to a linear support vector machine.

Original languageEnglish
Title of host publicationAdvances in Neural Networks - ISNN 2009 - 6th International Symposium on Neural Networks, ISNN 2009, Proceedings
Number of pages10
EditionPART 1
Publication statusPublished - 2009 Sept 11
Event6th International Symposium on Neural Networks, ISNN 2009 - Wuhan, China
Duration: 2009 May 262009 May 29

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 1
Volume5551 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference6th International Symposium on Neural Networks, ISNN 2009


  • Gender Classification
  • Neural Network
  • Rate Distortion Theory
  • Sparse Coding

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'An information theoretic perspective of the sparse coding'. Together they form a unique fingerprint.

Cite this