A robot singer with music recognition based on real-time beat tracking

Kazumasa Murata*, Kazuhiro Nakadai, Kazuyoshi Yoshii, Ryu Takeda, Toyotaka Torii, Hiroshi G. Okuno, Yuji Hasegawa, Hiroshi Tsujino

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)


A robot that can provide an active and enjoyable user interface is one of the most challenging applications for music information processing, because the robot should cope with high-power noises including self voices and motor noises. This paper proposes noise-robust musical beat tracking by using a robot-embedded microphone, and describes its application to a robot singer with music recognition. The proposed beat tracking introduces two key techniques, that is, spectro-temporal pattern matching and echo cancellation. The former realizes robust tempo estimation with a shorter window length, thus, it can quickly adapt to tempo changes. The latter is effective to cancel self periodic noises such as stepping, scatting, and singing. We constructed a robot singer based on the proposed beat tracking for Honda ASIMO. The robot detects a musical beat with its own microphone in a noisy environment. It tries to recognize music based on the detected musical beat. When it successfully recognizes music, it sings while stepping according to the beat. Otherwise, it performs scatting instead of singing because the lyrics are unavailable. Experimental results showed fast adaptation to tempo changes and high robustness in beat tracking even when stepping, scatting and singing.

Original languageEnglish
Title of host publicationISMIR 2008 - 9th International Conference on Music Information Retrieval
Number of pages6
Publication statusPublished - 2008
Externally publishedYes
Event9th International Conference on Music Information Retrieval, ISMIR 2008 - Philadelphia, PA
Duration: 2008 Sept 142008 Sept 18


Other9th International Conference on Music Information Retrieval, ISMIR 2008
CityPhiladelphia, PA

ASJC Scopus subject areas

  • Music
  • Information Systems


Dive into the research topics of 'A robot singer with music recognition based on real-time beat tracking'. Together they form a unique fingerprint.

Cite this