International Journal of Engineering and Manufacturing (IJEM)
IJEM Vol. 16, No. 4, 8 Aug. 2026
Cover page and Table of Contents: PDF (size: 1036KB)
Thermal Emotion Recognition with and without Synthetic Facial Images using Attention-Based EfficientNet5
PDF (1036KB), PP.211-227
Views: 0 Downloads: 0
Author(s)
Index Terms
Thermal Image, Facial Emotion Recognition, cGAN, Face Recognition, EfficientNetB5, CBAM
Abstract
Humans use emotions to express their feelings and effectively interact with others. Humans express emotions through hands, voice, gestures, and, most importantly, facial expressions. Facial emotion recognition is widely used in human-computer interaction, security, and healthcare. Traditional facial emotion recognition using visible light is often affected by changing lighting conditions. Thermal images could be used as an alternative solution because it relies on physiological heat patterns that remain consistent regardless of illumination. However, the use of thermal images for emotion recognition has not been extensively explored, primarily due to the scarcity of thermal image datasets. The study utilized the recently published Thermal Emotion dataset, which contains 2,250 thermal images before augmentation. After generating a synthetic dataset using cGAN, the datasets expanded to 6,823 images. To prevent dataset leakage, the study used 80% of the data for training, 10% for validation, and 10% for testing. The study used a bilateral filter to reduce noise while keeping relevant edge information, used CLAHE to enhance local contrast in low-intensity regions, making subtle thermal gradients more distinguishable, used Gaussian smoothing to reduce high-frequency noise, resulting in more stable feature extraction, and used attention mechanisms, CBAM, to allow the model to focus on emotion-relevant facial regions through its channel and spatial attention components. The combination of these techniques improved inter-class separability and contributed to measurable gains in recognition accuracy. The study findings show that inclusions of the bilateral filter, CLAHE cGAN, EfficientNetB5, and the CBAM attention mechanism significantly improved accuracy from 97.04% to 98.81%. For each experiment, ResNet18 classifies human facial emotions into five expressions: Happy, Sad, Angry, Natural, and Surprise. These findings imply that synthetic data generation using a cGAN to overcome data scarcity and an attention mechanism provides robust, lightning-independent solutions for facial emotion recognition.
Cite This Paper
Tahir Aman, Sintayehu Hirpassa, Kibreab Adane, "Thermal Emotion Recognition with and without Synthetic Facial Images using Attention-Based EfficientNet5", International Journal of Engineering and Manufacturing (IJEM), Vol.16, No.4, pp.211-227, 2026. DOI:10.5815/ijem.2026.04.15
Reference
[1]A. H. Krishna, A. B. Sri, K. Y. V. S. Priyanka, S. Taran, and V. Bajaj, “Emotion classification using EEG signals based on tunable-Q wavelet transform,” IET Science, Measurement and Technology, vol. 13, Iss. 3, pp. 375–380, 2019, doi: 10.1049/iet-smt.2018.5237
[2]E. S. Agung, A. P. Rifai, and T. Wijayanto, “Image-based facial emotion recognition using a convolutional neural network on Emotion Recognition Dataset,” Scientific Reports, vol. 14,14429 (2024), https://doi.org/10.1038/s41598-024-65276-x
[3]V. Jacintha, J. Simon, S. Tamilarasu, R. Thamizhmani, K. T. Yogesh, and J. Nagarajan, “A review on facial emotion recognition techniques,” in 2019 International Conference on Communication and Signal Processing (ICCSP), 2019, pp. 517–521, DOI: 10.1109/ICCSP.2019.8698067.
[4]E. G. Dada, D. O. Oyewola, S. B. Joseph, O. Emebo, and O. O. Oluwagbemi, “Facial emotion recognition and classification using the convolutional neural network-10 (CNN-10),” Applied Computational Intelligence and Soft Computing, vol. 2023, PP.19, 2023, DOI: https://doi.org/10.1155/2023/2457898
[5]A. Seal, S. Ganguly, D.Bhattacharjee, M. Nasipuri, and D.K.Basu, “Thermal Human Face Recognition Based on Haar Wavelet Transform and Series Matching Technique,” In: Swamy, P., Guru, D. (eds) Multimedia Processing, Communication and Computing Applications. Lecture Notes in Electrical Engineering, vol 213. Springer, New Delhi. https://doi.org/10.1007/978-81-322-1143-3_13
[6]A. Bhattacharyya, S. Chatterjee, S. Sen, A. Sinitca, D. Kaplun, and R. Sarkar, “A deep learning model for classifying human facial expressions from infrared thermal images,” Scientific Reports, vol. 11, 20696 (2021), https://doi.org/10.1038/s41598-021-99998-z
[7]Chaitanya, S. Sarath, Malavika, Prasanna, and Karthik, "Human Emotions Recognition from Thermal Images using Yolo Algorithm," 2020 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, 2020, pp. 1139-1142, doi: 10.1109/ICCSP48568.2020.9182148.
[8]A.K. Prabhakaran, J.J.Nair, S.Sarath, “Thermal facial expression recognition using modified ResNet152,” Advances in Computing and Network Communications, Lecture Notes in Electrical Engineering 736, 2021. https://doi.org/10.1007/978-981-33-6987-0_32
[9]B. Assiri and M. A. Hossain, “Face emotion recognition based on infrared thermal imagery by applying machine learning and parallelism,” Mathematical Biosciences and Engineering, vol. 20, pp. 913–929, 2023,doi: 10.3934/mbe.2023042.
[10]Yomna M. Elbarawy, Neveen I. Ghali, Rania Salah El-Sayed, " Facial Expressions Recognition in Thermal Images based on Deep Learning Techniques", International Journal of Image, Graphics and Signal Processing (IJIGSP), Vol.11, No.10, pp. 1-7, 2019. DOI: 10.5815/ijigsp.2019.10.01
[11]S. B. R. Prasad and B. S. Chandana, “Human face emotions recognition from thermal images using DenseNet,” International Journal of Electrical and Computer Engineering Systems, vol. 14, pp. 155–167, 2023, DOI: https://doi.org/10.32985/ijeces.14.2.5.
[12]C.K. Kyal, H.Poddar, H., M. Reza. “Human Emotion Recognition from Spontaneous Thermal Image Sequence Using GPU Accelerated Emotion Landmark Localization and Parallel Deep Emotion Net,” International Conference on Innovative Computing and Communications. Advances in Intelligent Systems and Computing, vol 1165,2020. Springer, Singapore. https://doi.org/10.1007/978-981-15-5113-0_78
[13]Abuhussein, Mohamed Fawzi; Darwish, Ashraf; Hassanien, Aboul Ella. “Comprehensive Facial Thermal Dataset”, Mendeley Data, V1,2024, doi: 10.17632/8885sc9p4z.1
[14]P. A. Flach, “The geometry of ROC space: Understanding machine learning metrics through ROC isometrics,” in Proceedings of the 20th International Conference on Machine Learning (ICML-03), Washington, DC, USA, 2003, pp. 194–201.
[15]H. Jiang, X. Chen, S. Chen, L. Ye, J. Hu and K. Yang, "Beyond the Visible: Deep Learning-Powered Thermal Face Recognition," in IEEE Internet of Things Journal, vol. 13, no. 7, pp. 14867-14881, 1 April1, 2026, doi: 10.1109/JIOT.2026.3652708.