Analysis of Emotion Certainty Formation in Dynamic Facial Expression Recognition

1 Hour

Facial Emotion Recognition (FER) has been widely studied in fields such as human-computer interaction, healthcare, and intelligent systems. However, many existing studies mainly evaluate the final recognition accuracy and do not sufficiently examine how a model gradually forms certainty about an emotion during a facial expression sequence.

This research analyzes the process of emotion certainty formation in dynamic facial expression recognition. A CNN-LSTM model is trained using the CK+ dataset to capture both spatial facial features and temporal changes across consecutive frames. Instead of focusing only on the final prediction, the study observes how the predicted emotion and its confidence change throughout the sequence.

To describe this temporal process, two recognition states are introduced: the Tentative State and the Confirmed State. In the Tentative State, the model has not yet obtained sufficient certainty to confirm an emotion. The Confirmed State represents the point at which the model determines that the emotion has become sufficiently clear. Based on these states, Trigger Time is defined as the time required for an emotion to first reach the Confirmed State.

By comparing Trigger Time across different emotion categories, this research aims to clarify differences in how quickly emotional certainty is formed. The results may contribute to a better understanding of the temporal decision-making process of dynamic FER models and improve their interpretability.