Jianzeng Chen, Ningning Chen, Journal of Information Processing Systems Vol. 20, No. 4, pp. 535-549, Aug. 2024  

https://doi.org/10.3745/JIPS.01.0107
Keywords: contextual information, Convolutional Channel Attention, Deep Learning, facial expression recognition, feature fusion, Reverse Attention
Fulltext: PDF Full Text PubReader

Abstract

Facial expressions (FEs) serve as fundamental components for human emotion assessment and human computer interaction. Traditional convolutional neural networks tend to overlook valuable information during the FE feature extraction, resulting in suboptimal recognition rates. To address this problem, we propose a deep learning framework that incorporates hierarchical feature fusion, contextual data, and an attention mechanism for precise FE recognition. In our approach, we leveraged an enhanced VGGNet16 as the backbone network and introduced an improved group convolutional channel attention (GCCA) module in each block to emphasize the crucial expression features. A partial decoder was added at the end of the backbone network to facilitate the fusion of multilevel features for a comprehensive feature map. A reverse attention mechanism guides the model to refine details layer-by-layer while introducing contextual information and extracting richer expression features. To enhance feature distinguishability, we employed islanding loss in combination with softmax loss, creating a joint loss function. Using two open datasets, our experimental results demonstrated the effectiveness of our framework. Our framework achieved an average accuracy rate of 74.08% on the FER2013 dataset and 98.66% on the CK+ dataset, outperforming advanced methods in both recognition accuracy and stability.

Statistics
Cite this article