A Smart Glasses-Based Real-Time Micro-expressions Recognition System via Deep Neural Network

Siyu Xiong, Xuan Huang, Kiminori Sato, Bo Wu*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The rapid development of new communication technologies such as social software and email has brought convenience, but it has also increased the distance between people. This research aims to capture micro-expressions in real-time through smart glasses and uses deep learning technology to develop a real-time emotion recognition system based on RGB color values to improve interpersonal communication. We trained a multi-layer fully deep neural network (DNN) model using the CASME2 dataset to effectively learn the association between facial expressions and emotions. The experimental results of model show that the emotion classification accuracy of the system reaches 95% on several test samples. The system is adapted to run on smart glasses, the emotion recognition results are immediately fed back to the screen of the smart glasses and displayed to the user in the form of an emotion label. The experimental results of system show that the system can achieve accurate emotion recognition, help users better understand the psychological state of the communication object, and improve the communication environment and quality. In addition, the system can also store communication records and emotion analysis results to help users understand their own communication performance and changes in others’ emotions, thus improving communication skills and abilities.

Original languageEnglish
Title of host publicationGreen, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings
EditorsHai Jin, Zhiwen Yu, Chen Yu, Xiaokang Zhou, Zeguang Lu, Xianhua Song
PublisherSpringer Science and Business Media Deutschland GmbH
Pages191-205
Number of pages15
ISBN (Print)9789819998951
DOIs
Publication statusPublished - 2024
Event18th International Conference on Green, Pervasive, and Cloud Computing, GPC 2023 - Harbin, China
Duration: 2023 Sept 222023 Sept 24

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14504
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference18th International Conference on Green, Pervasive, and Cloud Computing, GPC 2023
Country/TerritoryChina
CityHarbin
Period23/9/2223/9/24

Keywords

  • Deep Learning
  • Improving Communication Skills
  • MediaPipe
  • Micro expression Analysis
  • Real-Time Emotion Labels
  • Smart Glasses

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'A Smart Glasses-Based Real-Time Micro-expressions Recognition System via Deep Neural Network'. Together they form a unique fingerprint.

Cite this