Extraction of relationships between learners' physiological information and learners' mental states by machine learning

Yoshimasa Tawatsuji; Tatsuro Uno; Keita Okazaki; Siyuan Fang; Tatsunori Matsui

Extraction of relationships between learners' physiological information and learners' mental states by machine learning

Yoshimasa Tawatsuji, Tatsuro Uno, Keita Okazaki, Siyuan Fang, Tatsunori Matsui

Global Education Center

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.

Original language	English
Title of host publication	Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings
Editors	Ahmad Fauzi Mohd Ayub, Antonija Mitrovic, Jie-Chi Yang, Su Luan Wong, Wenli Chen
Publisher	Asia-Pacific Society for Computers in Education
Pages	56-61
Number of pages	6
ISBN (Print)	9789869401265
Publication status	Published - 2017 Jan 1
Event	25th International Conference on Computers in Education, ICCE 2017 - Christchurch, New Zealand Duration: 2017 Dec 4 → 2017 Dec 8

Other

Other	25th International Conference on Computers in Education, ICCE 2017
Country/Territory	New Zealand
City	Christchurch
Period	17/12/4 → 17/12/8

Keywords

Deep learning
Intelligent Mentoring System
Mental state estimation
Physiological data

ASJC Scopus subject areas

Computer Science (miscellaneous)
Computer Science Applications
Information Systems
Hardware and Architecture
Education

Cite this

Tawatsuji, Y., Uno, T., Okazaki, K., Fang, S., & Matsui, T. (2017). Extraction of relationships between learners' physiological information and learners' mental states by machine learning. In A. F. Mohd Ayub, A. Mitrovic, J-C. Yang, S. L. Wong, & W. Chen (Eds.), Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings (pp. 56-61). Asia-Pacific Society for Computers in Education.

Extraction of relationships between learners' physiological information and learners' mental states by machine learning. / Tawatsuji, Yoshimasa; Uno, Tatsuro; Okazaki, Keita et al.
Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. ed. / Ahmad Fauzi Mohd Ayub; Antonija Mitrovic; Jie-Chi Yang; Su Luan Wong; Wenli Chen. Asia-Pacific Society for Computers in Education, 2017. p. 56-61.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Tawatsuji, Y, Uno, T, Okazaki, K, Fang, S & Matsui, T 2017, Extraction of relationships between learners' physiological information and learners' mental states by machine learning. in AF Mohd Ayub, A Mitrovic, J-C Yang, SL Wong & W Chen (eds), Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. Asia-Pacific Society for Computers in Education, pp. 56-61, 25th International Conference on Computers in Education, ICCE 2017, Christchurch, New Zealand, 17/12/4.

Tawatsuji Y, Uno T, Okazaki K, Fang S, Matsui T. Extraction of relationships between learners' physiological information and learners' mental states by machine learning. In Mohd Ayub AF, Mitrovic A, Yang J-C, Wong SL, Chen W, editors, Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. Asia-Pacific Society for Computers in Education. 2017. p. 56-61

Tawatsuji, Yoshimasa ; Uno, Tatsuro ; Okazaki, Keita et al. / Extraction of relationships between learners' physiological information and learners' mental states by machine learning. Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings. editor / Ahmad Fauzi Mohd Ayub ; Antonija Mitrovic ; Jie-Chi Yang ; Su Luan Wong ; Wenli Chen. Asia-Pacific Society for Computers in Education, 2017. pp. 56-61

@inproceedings{7649715f2de840138c141389c164ddb4,

title = "Extraction of relationships between learners' physiological information and learners' mental states by machine learning",

abstract = "The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.",

keywords = "Deep learning, Intelligent Mentoring System, Mental state estimation, Physiological data",

author = "Yoshimasa Tawatsuji and Tatsuro Uno and Keita Okazaki and Siyuan Fang and Tatsunori Matsui",

year = "2017",

month = jan,

day = "1",

language = "English",

isbn = "9789869401265",

pages = "56--61",

editor = "{Mohd Ayub}, {Ahmad Fauzi} and Antonija Mitrovic and Jie-Chi Yang and Wong, {Su Luan} and Wenli Chen",

booktitle = "Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings",

publisher = "Asia-Pacific Society for Computers in Education",

note = "25th International Conference on Computers in Education, ICCE 2017 ; Conference date: 04-12-2017 Through 08-12-2017",

}

TY - GEN

T1 - Extraction of relationships between learners' physiological information and learners' mental states by machine learning

AU - Tawatsuji, Yoshimasa

AU - Uno, Tatsuro

AU - Okazaki, Keita

AU - Fang, Siyuan

AU - Matsui, Tatsunori

PY - 2017/1/1

Y1 - 2017/1/1

N2 - The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.

AB - The estimation of learners' mental states during the interaction between teachers and learners is a very important problem in improving the quality of teaching and learning. In this experimental study, we developed a deep learning neural network (DLNN) system that extracted the relationships between a learner's mental states and a teacher's utterances plus the learner's physiological information. The learner's physiological information consisted of the NIRS signals, the EEG signals, respiration intensity, skin conductance, and pulse volume. The learner's mental states were elicited through the learner's introspective reports using the Achievement Emotions Questionnaire (AEQ). According to the AEQ, the learner's mental states were divided into nine categories: Enjoy, Hope, Pride, Anger, Anxiety, Shame, Hopelessness, Boredom, and Others. In a simulation, the DLNN system exhibited the ability to estimate the learner's mental states from the learner's physiological information with high accuracy.

KW - Deep learning

KW - Intelligent Mentoring System

KW - Mental state estimation

KW - Physiological data

UR - http://www.scopus.com/inward/record.url?scp=85053881748&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053881748&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85053881748

SN - 9789869401265

SP - 56

EP - 61

BT - Proceedings of the 25th International Conference on Computers in Education, ICCE 2017 - Main Conference Proceedings

A2 - Mohd Ayub, Ahmad Fauzi

A2 - Mitrovic, Antonija

A2 - Yang, Jie-Chi

A2 - Wong, Su Luan

A2 - Chen, Wenli

PB - Asia-Pacific Society for Computers in Education

T2 - 25th International Conference on Computers in Education, ICCE 2017

Y2 - 4 December 2017 through 8 December 2017

ER -

Extraction of relationships between learners' physiological information and learners' mental states by machine learning

Abstract

Other

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this