Speech recognition technology combined with three dimensional lip movement

K. Komiya*, R. Ishikawa, K. Momose

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review


In order to improve speech recognition efficiency under background noise such as out of doors, we propose a new recognition technology combined with three dimensional lip movements. In this paper firstly, three-dimensional movements at four positions of the mouth were measured using principal component analysis to clarify which positions and which directions are main contributors to pronunciation. Secondly, recognition evaluation tests for 50 Japanese words were carried out under noise levels ranging from 40 to 80 dB. In the experiment, over 80% of recognition efficiency was measured at 70dB and improvement of 40% was obtained compared with ordinary speech recognition. From the experimental results, the proposed method can be modified to be used as practical speech recognition technology. Finally, research subjects were picked up such as an improvement in precision of measuring lip movement and experiments and data collection out of doors.

Original languageEnglish
Pages (from-to)95-102
Number of pages8
JournalProceedings of SPIE - The International Society for Optical Engineering
Publication statusPublished - 2001 Jan 1
Externally publishedYes
EventThree-Dimensional Image Capture and Applications IV - San Jose, CA, United States
Duration: 2001 Jan 242001 Jan 25


  • Background noise
  • Principal component analysis
  • Recognition efficiency
  • Three dimensional lip movement

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Speech recognition technology combined with three dimensional lip movement'. Together they form a unique fingerprint.

Cite this