TY - GEN
T1 - Audiovisual semantic congruency effect with onomatopoeia
AU - Fidalgo, Antonio Rei
AU - Takahashi, Kohshe
AU - Murata, Aiko
AU - Watanabe, Katsumi
N1 - Funding Information:
ACKNOWLEDGMENT This study was partly supported by a Japan Society for the Promotion of Science post-doctoral fellowship to ARF and; CREST, Japan Science and Technology Agency and KAKENHI grants (24300279 to KW and 25700013 to KT). The sponsors played no role in study design, data collection and analyses and interpretation of the data, writing and submission of the article.
Publisher Copyright:
© 2018 IEEE.
PY - 2018/8/6
Y1 - 2018/8/6
N2 - It has been reported that when a congruent natural sound precedes briefly presented visual stimuli, it promotes performance in psychophysics detection tasks. Onomatopoeias refer to words that phonetically mimic or suggest actual sounds. Onomatopoeic words are a form of sound symbolism and are frequently used in Japanese language. In this study, we examined whether the presentation of spoken Japanese onomatopoeia to Japanese native-speakers results in visual detection sensitivity changes. Results indicate that when onomatopoeias are presented 227 ms before a visual stimulus, they have a modulatory audiovisual effect. This effect is closer to the results observed with natural sounds than spoken words, with d' being lower for onomatopoeias when compared with natural sounds. Such suggests that Japanese spoken onomatopoeias may be processed in a manner that is closer to natural sounds than spoken words and points to behavioral consequences of sound symbolism.
AB - It has been reported that when a congruent natural sound precedes briefly presented visual stimuli, it promotes performance in psychophysics detection tasks. Onomatopoeias refer to words that phonetically mimic or suggest actual sounds. Onomatopoeic words are a form of sound symbolism and are frequently used in Japanese language. In this study, we examined whether the presentation of spoken Japanese onomatopoeia to Japanese native-speakers results in visual detection sensitivity changes. Results indicate that when onomatopoeias are presented 227 ms before a visual stimulus, they have a modulatory audiovisual effect. This effect is closer to the results observed with natural sounds than spoken words, with d' being lower for onomatopoeias when compared with natural sounds. Such suggests that Japanese spoken onomatopoeias may be processed in a manner that is closer to natural sounds than spoken words and points to behavioral consequences of sound symbolism.
KW - Audiovisual interaction
KW - Onomatopoeia
KW - Semantic congruency
KW - Spoken word
KW - Visual detection
UR - http://www.scopus.com/inward/record.url?scp=85052287411&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85052287411&partnerID=8YFLogxK
U2 - 10.1109/KST.2018.8426115
DO - 10.1109/KST.2018.8426115
M3 - Conference contribution
AN - SCOPUS:85052287411
SN - 9781538640159
T3 - 2018 10th International Conference on Knowledge and Smart Technology: Cybernetics in the Next Decades, KST 2018
SP - 254
EP - 259
BT - 2018 10th International Conference on Knowledge and Smart Technology
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 10th International Conference on Knowledge and Smart Technology, KST 2018
Y2 - 31 January 2018 through 3 February 2018
ER -