Title :

Improvement of emotion recognition from facial images using deep learning and early stopping cross validation

Author(s) :

Bentoumi, Mohamed [Auteur]
Daoud, Mohamed [Auteur]
Benaouali, Mohamed [Auteur]
Tahleb Ahmed, Abdelmalik [Auteur]
COMmunications NUMériques - IEMN [COMNUM - IEMN]
Institut d’Électronique, de Microélectronique et de Nanotechnologie - UMR 8520 [IEMN]

Journal title :

Multimedia Tools and Applications

Pages :

29887-29917

Publisher :

Springer Verlag

Publication date :

2022-09

ISSN :

1380-7501

English keyword(s) :

Emotion recognition
Facial images
Pretrained CNN
Features extraction
MLP classifier
k-fold cross validation
Early stopping
Classification performance metrics

HAL domain(s) :

Sciences de l'ingénieur [physics]

English abstract : [en]

Inthis paper, we present a new approach for emotion recognition from facial images. The proposed method is based on the association of a pretrained convolutional neural network (CNN) model (VGG16, ResNet50) with a multilayer ...
Show more >Inthis paper, we present a new approach for emotion recognition from facial images. The proposed method is based on the association of a pretrained convolutional neural network (CNN) model (VGG16, ResNet50) with a multilayer perceptron (MLP) classifier. The pretrained CNN model is used as a feature extractor. For this purpose, we adapt the original architecture by adding a global average pooling layer (GAP) without any fine tuning of the network parameters. In order to avoid overfitting for the MLP classifier, we introduce the early stopping criterion. It is proved that the aforementioned elements contribute in improving the performance of our approach in terms of generalization ability. The procedure for emotion recognition from facial images is applied on the CK+ (extended Cohen-Kanad), JAFFE (Japanese Female Facial Expression) and KDEF (Karolinska Directed Emotional Faces) databases. The k-fold cross validation procedure is used for accuracy estimation. The experimental results show the effectiveness of our facial emotion recognition (FER) approach compared to the existing methods yielding to recognition rates of 100%, 96.40% and 98.78% for the CK+, JAFFE and KDEF databases, respectively. On the other hand, further improvement of our recognition performance is obtained for images from the JAFFE database by performing a data augmentation during the training phase. This allows to achieve an accuracy of 100% for this database. Four other metrics, namely the F1-score, positive and negative likelihood ratios and Mathews correlation coefficient, confirm as well the classification results obtained from accuracy.Show less >

Language :

Anglais

Popular science :

Non

Collections :

• Institut d'Électronique, de Microélectronique et de Nanotechnologie (IEMN) - UMR 8520

Source :

Harvested from HAL