An open source software system for robot audition HARK and its evaluation

Kazuhiro Nakadai; Hiroshi G. Okuno; Hirofumi Nakajima; Yuji Hasegawa; Hiroshi Tsujino

doi:10.1109/ICHR.2008.4756031

An open source software system for robot audition HARK and its evaluation

Kazuhiro Nakadai^*, Hiroshi G. Okuno, Hirofumi Nakajima, Yuji Hasegawa, Hiroshi Tsujino

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

56 Citations (Scopus)

Abstract

Robot capability of listening to several things at once by its own ears, that is, robot audition, is important in improving human-robot interaction. The critical issue in robot audition is real-time processing in noisy environments with high flexibility to support various kinds of robots and hardware configurations. This paper presents open-source robot audition software, called "HARK", which includes sound source localization, separation, and automatic speech recognition (ASR). Since separated sounds suffer from spectral distortion due to separation, HARK generates a temporal-frequency map of reliability, called "missing feature mask", for features of separated sounds. Then separated sounds are recognized by the Missing-Feature Theory (MFT) based ASR with missing feature masks. HARK is implemented on the middleware called "FlowDesigner" to share intermediate audio data, which provides real-time processing. HARK's performance in recognition of noisy/simultaneous speech is shown by using three humanoid robots, Honda ASIMO, SIG2 and Robovie with different microphone layouts.

Original language	English
Title of host publication	2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008
Pages	561-566
Number of pages	6
DOIs	https://doi.org/10.1109/ICHR.2008.4756031
Publication status	Published - 2008
Externally published	Yes
Event	2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008 - Daejeon Duration: 2008 Dec 1 → 2008 Dec 3

Other

Other	2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008
City	Daejeon
Period	08/12/1 → 08/12/3

ASJC Scopus subject areas

Artificial Intelligence
Computer Vision and Pattern Recognition
Human-Computer Interaction

Access to Document

10.1109/ICHR.2008.4756031

Cite this

Nakadai, K, Okuno, HG, Nakajima, H, Hasegawa, Y & Tsujino, H 2008, An open source software system for robot audition HARK and its evaluation. in 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008., 4756031, pp. 561-566, 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008, Daejeon, 08/12/1. https://doi.org/10.1109/ICHR.2008.4756031

@inproceedings{ee1e3abd8a8b402bbaa3748a7aa761ab,

title = "An open source software system for robot audition HARK and its evaluation",

abstract = "Robot capability of listening to several things at once by its own ears, that is, robot audition, is important in improving human-robot interaction. The critical issue in robot audition is real-time processing in noisy environments with high flexibility to support various kinds of robots and hardware configurations. This paper presents open-source robot audition software, called {"}HARK{"}, which includes sound source localization, separation, and automatic speech recognition (ASR). Since separated sounds suffer from spectral distortion due to separation, HARK generates a temporal-frequency map of reliability, called {"}missing feature mask{"}, for features of separated sounds. Then separated sounds are recognized by the Missing-Feature Theory (MFT) based ASR with missing feature masks. HARK is implemented on the middleware called {"}FlowDesigner{"} to share intermediate audio data, which provides real-time processing. HARK's performance in recognition of noisy/simultaneous speech is shown by using three humanoid robots, Honda ASIMO, SIG2 and Robovie with different microphone layouts.",

author = "Kazuhiro Nakadai and Okuno, {Hiroshi G.} and Hirofumi Nakajima and Yuji Hasegawa and Hiroshi Tsujino",

year = "2008",

doi = "10.1109/ICHR.2008.4756031",

language = "English",

isbn = "9781424428229",

pages = "561--566",

booktitle = "2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008",

note = "2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008 ; Conference date: 01-12-2008 Through 03-12-2008",

}

TY - GEN

T1 - An open source software system for robot audition HARK and its evaluation

AU - Nakadai, Kazuhiro

AU - Okuno, Hiroshi G.

AU - Nakajima, Hirofumi

AU - Hasegawa, Yuji

AU - Tsujino, Hiroshi

PY - 2008

Y1 - 2008

N2 - Robot capability of listening to several things at once by its own ears, that is, robot audition, is important in improving human-robot interaction. The critical issue in robot audition is real-time processing in noisy environments with high flexibility to support various kinds of robots and hardware configurations. This paper presents open-source robot audition software, called "HARK", which includes sound source localization, separation, and automatic speech recognition (ASR). Since separated sounds suffer from spectral distortion due to separation, HARK generates a temporal-frequency map of reliability, called "missing feature mask", for features of separated sounds. Then separated sounds are recognized by the Missing-Feature Theory (MFT) based ASR with missing feature masks. HARK is implemented on the middleware called "FlowDesigner" to share intermediate audio data, which provides real-time processing. HARK's performance in recognition of noisy/simultaneous speech is shown by using three humanoid robots, Honda ASIMO, SIG2 and Robovie with different microphone layouts.

AB - Robot capability of listening to several things at once by its own ears, that is, robot audition, is important in improving human-robot interaction. The critical issue in robot audition is real-time processing in noisy environments with high flexibility to support various kinds of robots and hardware configurations. This paper presents open-source robot audition software, called "HARK", which includes sound source localization, separation, and automatic speech recognition (ASR). Since separated sounds suffer from spectral distortion due to separation, HARK generates a temporal-frequency map of reliability, called "missing feature mask", for features of separated sounds. Then separated sounds are recognized by the Missing-Feature Theory (MFT) based ASR with missing feature masks. HARK is implemented on the middleware called "FlowDesigner" to share intermediate audio data, which provides real-time processing. HARK's performance in recognition of noisy/simultaneous speech is shown by using three humanoid robots, Honda ASIMO, SIG2 and Robovie with different microphone layouts.

UR - http://www.scopus.com/inward/record.url?scp=63549118078&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=63549118078&partnerID=8YFLogxK

U2 - 10.1109/ICHR.2008.4756031

DO - 10.1109/ICHR.2008.4756031

M3 - Conference contribution

AN - SCOPUS:63549118078

SN - 9781424428229

SP - 561

EP - 566

BT - 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008

T2 - 2008 8th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2008

Y2 - 1 December 2008 through 3 December 2008

ER -

An open source software system for robot audition HARK and its evaluation

Abstract

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this