Constructing a Model for Estimating Learners’ Mental States Using Biometric Information and Reducing Labeling Costs Using Deep Learning

In the teaching and learning process, it is important to understand not only the state of understanding but also the mental state of the learner. However, the mental state of the learner is not always expressed in facial expressions or movements, and it is sometimes difficult to observe or measure from the outside. Therefore, previous studies have tried to estimate the mental state by focusing on biometric information. However, previous studies have ignored the time-series nature of the data, resulting in a model that is difficult to generalize. In addition, the labeling cost of subjects in the previous studies was large, making the model unrealistic from the perspective of widespread use of educational systems. In this study, we experimentally investigated the generalizability of the estimation and the reduction of the labeling cost using deep learning. As a result, we found that the emotion decay model proposed in this study and its response to a small number of samples contributed the most to generalizability. We also confirmed that the combination of these findings with domain adaptation could reduce the labeling cost by up to 80%.