Understanding nonverbal communication cues of human personality traits in human-robot interaction

Zhihao Shen*, Armagan Elibol, Nak Young Chong

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

20 Citations (Scopus)


With the increasing presence of robots in our daily life, there is a strong need and demand for the strategies to acquire a high quality interaction between robots and users by enabling robots to understand users'mood, intention, and other aspects. During human-human interaction, personality traits have an important influence on human behavior, decision, mood, and many others. Therefore, we propose an efficient computational framework to endow the robot with the capability of understanding the user's personality traits based on the user's nonverbal communication cues represented by three visual features including the head motion, gaze, and body motion energy, and three vocal features including voice pitch, voice energy, and mel-frequency cepstral coefficient MFCC. We used the Pepper robot in this study as a communication robot to interact with each participant by asking questions, and meanwhile, the robot extracts the nonverbal features from each participant's habitual behavior using its on-board sensors. On the other hand, each participant's personality traits are evaluated with a questionnaire. We then train the ridge regression and linear support vector machine SVM classifiers using the nonverbal features and personality trait labels from a questionnaire and evaluate the performance of the classifiers. We have verified the validity of the proposed models that showed promising binary classification performance on recognizing each of the Big Five personality traits of the participants based on individual differences in nonverbal communication cues.

Original languageEnglish
Article number9106874
Pages (from-to)1465-1477
Number of pages13
JournalIEEE/CAA Journal of Automatica Sinica
Issue number6
Publication statusPublished - 2020 Nov
Externally publishedYes

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Information Systems
  • Artificial Intelligence


Dive into the research topics of 'Understanding nonverbal communication cues of human personality traits in human-robot interaction'. Together they form a unique fingerprint.

Cite this