Real-time emotion recognition system with multiple physiological signals

Jyun Rong Zhuang, Ya Jing Guan, Hayato Nagayoshi, Keiichi Muramatsu, Keiichi Watanuki, Eiichiro Tanaka*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

12 Citations (Scopus)


Emotion is an internal and subjective experience that plays a significant role in human life. There are several methods of recognizing emotions in people, the most authentic of which is using physiological signals, as they are beyond one’s control and strongly correlated with human emotions. This study aims to develop an emotion recognition system based on three physiological signals, namely, brainwave, heartbeat, and facial muscular activity. It utilizes deep neural network (DNN) and the T method of Mahalanobis-Taguchi system (MTS) to process the multiple physiological signals and further recognize the states of human emotion. As such, nine emotions are effectively recognized on a two-dimensional model through the DNN, then compared against several other algorithms, such as MTS, SVM, Naive Bayes, and K-means, where its superior accuracy is validated. Moreover, although the T method only improves the classification accuracy on the valence state, it rather obtains the intensity of emotion in different states. Furthermore, in this study, the proposed DNN is implemented into a wide range of applications for an accurate understanding of the human emotional states, whereas the T method is utilized to respond to the emotional intensity in different states. Finally, a real-time emotion recognition system is developed with DNN as the classifier; this system can directly monitor the variation of the human emotion through reliable and objective emotion analysis results from the physiological signals. Thus, the method can provide useful treatment effect information for robots or assistive apparatus serving activities of daily living.

Original languageEnglish
JournalJournal of Advanced Mechanical Design, Systems and Manufacturing
Issue number4
Publication statusPublished - 2019


  • Deep neural networks
  • Emotional evaluation
  • Multiple physiological signals
  • Real-time emotion recognition system
  • Two-dimensional emotion model

ASJC Scopus subject areas

  • Mechanical Engineering
  • Industrial and Manufacturing Engineering


Dive into the research topics of 'Real-time emotion recognition system with multiple physiological signals'. Together they form a unique fingerprint.

Cite this