Making a robot recognize three simultaneous sentences in real-time

Shun'Ichi Yamamoto*, Kazuhiro Nakadai, Jean Marc Valin, Jean Rouat, François Michaud, Kazunori Komatani, Tetsuya Ogata, Hiroshi G. Okuno


研究成果: Conference contribution

31 被引用数 (Scopus)


A humanoid robot under real-world environments usually hears mixtures of sounds, and thus three capabilities are essential for robot audition; sound source localization, separation, and recognition of separated sounds. We have adopted the missing feature theory (MFT) for automatic recognition of separated speech, and developed the robot audition system. A microphone array is used along with a real-time dedicated implementation of Geometric Source Separation (GSS) and a multi-channel post-filter that gives us a further reduction of interferences from other sources. The automatic speech recognition based on MFT recognizes separated sounds by generating missing feature masks automatically from the post-filtering step. The main advantage of this approach for humanoid robots resides in the fact that the ASR with a clean acoustic model can adapt the distortion of separated sound by consulting the post-filter feature masks. In this paper, we used the improved Julius as an MFT-based automatic speech recognizer (ASR). The Julius is a real-time large vocabulary continuous speech recognition (LVCSR) system. We performed the experiment to evaluate our robot audition system. In this experiment, the system recognizes a sentence, not an isolated word. We showed the improvement in the system performance through three simultaneous speech recognition on the humanoid SIG2.

ホスト出版物のタイトル2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS
出版社IEEE Computer Society
ISBN(印刷版)0780389123, 9780780389120
出版ステータスPublished - 2005


名前2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS

ASJC Scopus subject areas

  • 人工知能
  • コンピュータ ビジョンおよびパターン認識
  • 人間とコンピュータの相互作用
  • 制御およびシステム工学


「Making a robot recognize three simultaneous sentences in real-time」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。