TY - GEN
T1 - Bowing-Net
T2 - Special Interest Group on Computer Graphics and Interactive Techniques Conference: Posters, SIGGRAPH 2021
AU - Hirata, Asuka
AU - Tanaka, Keitaro
AU - Shimamura, Ryo
AU - Morishima, Shigeo
N1 - Funding Information:
This work was supported by JST Mirai Program No. JPMJMI19B2, and JSPS KAKENHI Nos. 19H01129 and 19H04137.
Publisher Copyright:
© 2021 Owner/Author.
PY - 2021/8/5
Y1 - 2021/8/5
N2 - This paper presents a deep learning based method that generates body motion for string instrument performance from raw audio. In contrast to prior methods which aim to predict joint position from audio, we first estimate information that dictates the bowing dynamics, such as the bow direction and the played string. The final body motion is then determined from this information following a conversion rule. By adopting the bowing information as the target domain, not only is learning the mapping more feasible, but also the produced results have bowing dynamics that are consistent with the given audio. We confirmed that our results are superior to existing methods through extensive experiments.
AB - This paper presents a deep learning based method that generates body motion for string instrument performance from raw audio. In contrast to prior methods which aim to predict joint position from audio, we first estimate information that dictates the bowing dynamics, such as the bow direction and the played string. The final body motion is then determined from this information following a conversion rule. By adopting the bowing information as the target domain, not only is learning the mapping more feasible, but also the produced results have bowing dynamics that are consistent with the given audio. We confirmed that our results are superior to existing methods through extensive experiments.
KW - Motion generation
KW - music information retrieval
KW - neural networks.
UR - http://www.scopus.com/inward/record.url?scp=85112730228&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85112730228&partnerID=8YFLogxK
U2 - 10.1145/3450618.3469170
DO - 10.1145/3450618.3469170
M3 - Conference contribution
AN - SCOPUS:85112730228
T3 - Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters, SIGGRAPH 2021
BT - Special Interest Group on Computer Graphics and Interactive Techniques Conference Posters, SIGGRAPH 2021
PB - Association for Computing Machinery, Inc
Y2 - 9 August 2021 through 13 August 2021
ER -