Rhythmic body movement analysis for robot-based music therapy

Y. H. Ma, J. Y. Lin, S. Cosentino*, A. Takanishi


研究成果: Conference contribution

2 被引用数 (Scopus)


The ability to correctly perceive time and extract accurate timing information is crucial during social interaction. In fact, several activities during social interaction, such as appropriate feedback, turn-taking, coordination with peers, and even empathy and engagement exhibition directly depend on it. One of the aspects of cognitive malfunctioning in children with Autistic Spectrum Disorders is time perception deficit. Learning to pay attention to and correctly assess timing is thus a critical first step to improve social skills for children with Autism. In this paper, we present a novel sensing system and algorithm for estimating a subject's rhythmic motion timing from visual information using Recurrent Neural Network (RNN) coupled with FFT. This system will enable a robot saxophonist to estimate the rhythmic period from a child's motion during a robot-based music therapy session. Fast-Fourier- Transform (FFT) is an algorithm widely applied in rhythmic body movement detection, due to advantages such low computation and easy integration. However, long transient time delay is a critical limitation, reducing the correct motion timing estimation during period transitions. The novel system presented in this article is shown to significantly reduce transient time delay. The results of both a simulation and an evaluation experiment show that, compared with FFT processing alone, this algorithm gives a better performance due to its smaller average offset error and shorter transient time delay, allowing a more precise assessment of the child's synchronization response.

ホスト出版物のタイトル2021 IEEE International Conference on Advanced Robotics and Its Social Impacts, ARSO 2021
出版社IEEE Computer Society
出版ステータスPublished - 2021 7月 8
イベント2021 IEEE International Conference on Advanced Robotics and Its Social Impacts, ARSO 2021 - Tokoname, Japan
継続期間: 2021 7月 82021 7月 10


名前Proceedings of IEEE Workshop on Advanced Robotics and its Social Impacts, ARSO


Conference2021 IEEE International Conference on Advanced Robotics and Its Social Impacts, ARSO 2021

ASJC Scopus subject areas

  • 人工知能
  • コンピュータ サイエンスの応用
  • コンピュータ ビジョンおよびパターン認識
  • 電子工学および電気工学


「Rhythmic body movement analysis for robot-based music therapy」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。