Voice articulatory training with a talking robot for the auditory impaired

Mitsuki Kitani*, Tatusya Hara, Hideyuki Sawada

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


The authors have developed a vocalization training system for the auditory impaired using a talking robot. The training system mainly consists of a talking robot that has mechanical organs like a human. With an adaptive learning strategy using an auditory feedback control, the robot autonomously learns the vocalization, and then reproduces the speech articulation from inputted sounds. In the previous system, the speech-learning algorithm of the robot was constructed by employing a self-organizing neural network (SONN), which consists of the combination of a self-organizing map (SOM) and a neural network (NN). However, improper maps were occasionally generated in the results of the speech articulation learning. To solve this problem, a new algorithm introducing two three-dimensional SOMs, called a dual-SOM, was employed for the autonomous learning of the robotic articulations. By applying the robot and its properties, we have constructed an interactive training system. The training is divided into two approaches; one is to use the talking robot to show the shape and the motion of the vocal organs, and the other is to use a topological map to present the difference of phonetic features of a trainee's voices. In this study, first, the construction of the training system is described together with the autonomous learning of robotic vocalization using the dual-SOM algorithm, and then the analysis of the speech training progress is presented based on the phonetic features and the mechanical vocal articulations.

Original languageEnglish
Pages (from-to)63-67
Number of pages5
JournalInternational Journal on Disability and Human Development
Issue number1
Publication statusPublished - 2011 Mar
Externally publishedYes


  • Adaptive learning
  • Interactive speech training system
  • Talking robot

ASJC Scopus subject areas

  • Rehabilitation
  • Sensory Systems
  • Geriatrics and Gerontology
  • Psychiatry and Mental health
  • Advanced and Specialised Nursing
  • Speech and Hearing


Dive into the research topics of 'Voice articulatory training with a talking robot for the auditory impaired'. Together they form a unique fingerprint.

Cite this