Audio-based automatic generation of a piano reduction score by considering the musical structure

Hirofumi Takamori*, Takayuki Nakatsuka, Satoru Fukayama, Masataka Goto, Shigeo Morishima

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)


This study describes a method that automatically generates a piano reduction score from the audio recordings of popular music while considering the musical structure. The generated score comprises both right- and left-hand piano parts, which reflect the melodies, chords, and rhythms extracted from the original audio signals. Generating such a reduction score from an audio recording is challenging because automatic music transcription is still considered to be inefficient when the input contains sounds from various instruments. Reflecting the long-term correlation structure behind similar repetitive bars is also challenging; further, previous methods have independently generated each bar. Our approach addresses the aforementioned issues by integrating musical analysis, especially structural analysis, with music generation. Our method extracts rhythmic features as well as melodies and chords from the input audio recording and reflects them in the score. To consider the long-term correlation between bars, we use similarity matrices, created for several acoustical features, as constraints. We further conduct a multivariate regression analysis to determine the acoustical features that represent the most valuable constraints for generating a musical structure. We have generated piano scores using our method and have observed that we can produce scores that differently balance between the ability to achieve rhythmic characteristics and the ability to obtain musical structures.

Original languageEnglish
Title of host publicationMultiMedia Modeling - 25th International Conference, MMM 2019, Proceedings
EditorsBenoit Huet, Ioannis Kompatsiaris, Stefanos Vrochidis, Vasileios Mezaris, Wen-Huang Cheng, Cathal Gurrin
PublisherSpringer Verlag
Number of pages13
ISBN (Print)9783030057152
Publication statusPublished - 2019
Event25th International Conference on MultiMedia Modeling, MMM 2019 - Thessaloniki, Greece
Duration: 2019 Jan 82019 Jan 11

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11296 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other25th International Conference on MultiMedia Modeling, MMM 2019


  • Acoustic feature
  • Multivariate regression analysis
  • Musical structure
  • Piano reduction
  • Self-similarity matrix

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Audio-based automatic generation of a piano reduction score by considering the musical structure'. Together they form a unique fingerprint.

Cite this