A musical mood trajectory estimation method using lyrics and acoustic features

Naoki Nishikawa; Katsutoshi Itoyama; Hiromasa Fujihara; Masataka Goto; Tetsuya Ogata; Hiroshi G. Okuno

doi:10.1145/2072529.2072543

A musical mood trajectory estimation method using lyrics and acoustic features

Naoki Nishikawa^*, Katsutoshi Itoyama, Hiromasa Fujihara, Masataka Goto, Tetsuya Ogata, Hiroshi G. Okuno

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

In this paper, we present a new method that represents an overall musical time-varying impression of a song by a pair of mood trajectories estimated from lyrics and audio signals. The mood trajectory of the lyrics is obtained by using the probabilistic latent semantic analysis (PLSA) to estimate topics (representing impressions) from words in the lyrics. The mood trajectory of the audio signals is estimated from acoustic features by using the multiple linear regression analysis. In our experiments, the mood trajectories of 100 songs in Last.fm's Best of 2010 were estimated. The detailed analysis of the 100 songs confirms that acoustic features provide more accurate mood trajectory and the 21% resulting mood trajectories are matched to realistic musical mood available at Last.fm.

Original language	English
Title of host publication	MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11
Pages	51-56
Number of pages	6
DOIs	https://doi.org/10.1145/2072529.2072543
Publication status	Published - 2011
Externally published	Yes
Event	2011 ACM Multimedia Conference, MM'11 and Co-Located Workshops - 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies, MIRUM'11 - Scottsdale, AZ, United States Duration: 2011 Nov 28 → 2011 Dec 1

Publication series

Name	MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11

Conference

Conference	2011 ACM Multimedia Conference, MM'11 and Co-Located Workshops - 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies, MIRUM'11
Country/Territory	United States
City	Scottsdale, AZ
Period	11/11/28 → 11/12/1

Keywords

Lyrics and audio signals music information retrieval
Mood trajectory
Musical mood estimation
Musical mood representation
Time-varying impression

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Human-Computer Interaction

Access to Document

10.1145/2072529.2072543

Cite this

Nishikawa, N., Itoyama, K., Fujihara, H., Goto, M., Ogata, T., & Okuno, H. G. (2011). A musical mood trajectory estimation method using lyrics and acoustic features. In MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11 (pp. 51-56). (MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11). https://doi.org/10.1145/2072529.2072543

A musical mood trajectory estimation method using lyrics and acoustic features. / Nishikawa, Naoki; Itoyama, Katsutoshi; Fujihara, Hiromasa et al.
MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11. 2011. p. 51-56 (MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Nishikawa, N, Itoyama, K, Fujihara, H, Goto, M, Ogata, T & Okuno, HG 2011, A musical mood trajectory estimation method using lyrics and acoustic features. in MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11. MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11, pp. 51-56, 2011 ACM Multimedia Conference, MM'11 and Co-Located Workshops - 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies, MIRUM'11, Scottsdale, AZ, United States, 11/11/28. https://doi.org/10.1145/2072529.2072543

Nishikawa N, Itoyama K, Fujihara H, Goto M, Ogata T, Okuno HG. A musical mood trajectory estimation method using lyrics and acoustic features. In MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11. 2011. p. 51-56. (MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11). doi: 10.1145/2072529.2072543

Nishikawa, Naoki ; Itoyama, Katsutoshi ; Fujihara, Hiromasa et al. / A musical mood trajectory estimation method using lyrics and acoustic features. MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11. 2011. pp. 51-56 (MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11).

@inproceedings{797e8f0fae884997854020b88b08bfda,

title = "A musical mood trajectory estimation method using lyrics and acoustic features",

abstract = "In this paper, we present a new method that represents an overall musical time-varying impression of a song by a pair of mood trajectories estimated from lyrics and audio signals. The mood trajectory of the lyrics is obtained by using the probabilistic latent semantic analysis (PLSA) to estimate topics (representing impressions) from words in the lyrics. The mood trajectory of the audio signals is estimated from acoustic features by using the multiple linear regression analysis. In our experiments, the mood trajectories of 100 songs in Last.fm's Best of 2010 were estimated. The detailed analysis of the 100 songs confirms that acoustic features provide more accurate mood trajectory and the 21% resulting mood trajectories are matched to realistic musical mood available at Last.fm.",

keywords = "Lyrics and audio signals music information retrieval, Mood trajectory, Musical mood estimation, Musical mood representation, Time-varying impression",

author = "Naoki Nishikawa and Katsutoshi Itoyama and Hiromasa Fujihara and Masataka Goto and Tetsuya Ogata and Okuno, {Hiroshi G.}",

year = "2011",

doi = "10.1145/2072529.2072543",

language = "English",

isbn = "9781450309868",

series = "MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11",

pages = "51--56",

booktitle = "MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11",

note = "2011 ACM Multimedia Conference, MM'11 and Co-Located Workshops - 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies, MIRUM'11 ; Conference date: 28-11-2011 Through 01-12-2011",

}

TY - GEN

T1 - A musical mood trajectory estimation method using lyrics and acoustic features

AU - Nishikawa, Naoki

AU - Itoyama, Katsutoshi

AU - Fujihara, Hiromasa

AU - Goto, Masataka

AU - Ogata, Tetsuya

AU - Okuno, Hiroshi G.

PY - 2011

Y1 - 2011

N2 - In this paper, we present a new method that represents an overall musical time-varying impression of a song by a pair of mood trajectories estimated from lyrics and audio signals. The mood trajectory of the lyrics is obtained by using the probabilistic latent semantic analysis (PLSA) to estimate topics (representing impressions) from words in the lyrics. The mood trajectory of the audio signals is estimated from acoustic features by using the multiple linear regression analysis. In our experiments, the mood trajectories of 100 songs in Last.fm's Best of 2010 were estimated. The detailed analysis of the 100 songs confirms that acoustic features provide more accurate mood trajectory and the 21% resulting mood trajectories are matched to realistic musical mood available at Last.fm.

AB - In this paper, we present a new method that represents an overall musical time-varying impression of a song by a pair of mood trajectories estimated from lyrics and audio signals. The mood trajectory of the lyrics is obtained by using the probabilistic latent semantic analysis (PLSA) to estimate topics (representing impressions) from words in the lyrics. The mood trajectory of the audio signals is estimated from acoustic features by using the multiple linear regression analysis. In our experiments, the mood trajectories of 100 songs in Last.fm's Best of 2010 were estimated. The detailed analysis of the 100 songs confirms that acoustic features provide more accurate mood trajectory and the 21% resulting mood trajectories are matched to realistic musical mood available at Last.fm.

KW - Lyrics and audio signals music information retrieval

KW - Mood trajectory

KW - Musical mood estimation

KW - Musical mood representation

KW - Time-varying impression

UR - http://www.scopus.com/inward/record.url?scp=84555188101&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84555188101&partnerID=8YFLogxK

U2 - 10.1145/2072529.2072543

DO - 10.1145/2072529.2072543

M3 - Conference contribution

AN - SCOPUS:84555188101

SN - 9781450309868

T3 - MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11

SP - 51

EP - 56

BT - MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11

T2 - 2011 ACM Multimedia Conference, MM'11 and Co-Located Workshops - 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies, MIRUM'11

Y2 - 28 November 2011 through 1 December 2011

ER -

A musical mood trajectory estimation method using lyrics and acoustic features

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this