Sun, Dec 22, 2024
[Archive]
Volume 15, Issue 4 (10-2023)
itrc 2023, 15(4): 25-31 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Khani Yengikand A, Farrokhi Afsharyan M, Nejati P. Facial Expression Recognition Based on Separable Convolution Network and Attention Mechanism. itrc 2023; 15 (4) : 31
URL: http://journal.itrc.ac.ir/article-1-566-en.html
URL: http://journal.itrc.ac.ir/article-1-566-en.html
1- Department of Computer Engineering, University of Zanjan, Zanjan, Iran , amirkhani@znu.ac.ir
2- Department of Electrical Engineering, Shahid Sattari Aeronautical University Tehran, Iran
3- Department of Electrical Engineering, Malek Ashtar University of Technology, Tehran, Iran
2- Department of Electrical Engineering, Shahid Sattari Aeronautical University Tehran, Iran
3- Department of Electrical Engineering, Malek Ashtar University of Technology, Tehran, Iran
Abstract: (1788 Views)
Facial expression recognition using deep learning methods has been one of the active research fields in the last decade. However, most of the previous works have focused on the implementation of the model in the laboratory environment, and few researchers have addressed the real-world challenges of facial expression recognition systems. One of the challenges of implementing the face recognition system in the real environment (e.g. webcam or robot) is to create a balance between accuracy and speed of model recognition. Because, increasing the complexity of the neural network model leads to an increase in the accuracy of the model, but due to the increase in the size of the model, the recognition speed of the model decreases. Therefore, in this paper, we propose a model to recognize the seven main emotions (Happiness, sadness, anger, surprise, fear, disgust and natural), which can create a balance between accuracy and recognition speed. Specifically, the proposed model has three main components. First, in the feature extraction component, the features of the input images are extracted using a combination of normal and separable convolutional networks. Second, in the feature integration component, the extracted features are integrated using the attention mechanism. Finally, the merged features are used as the input of the multi-layer perceptron neural network to recognize the input facial expression. Our proposed approach has been evaluated using three public datasets and images received via webcam
Article number: 31
Type of Study: Research |
Subject:
Information Technology
References
1. [1] Li, S. and W. Deng, Deep facial expression recognition: A survey. IEEE transactions on affective computing, 2020.
2. [2] Pramerdorfer, C. and M. Kampel, Facial expression recognition using convolutional neural networks: state of the art. arXiv preprint arXiv:1612.02903, 2016.
3. [3] Yengikand, A.K., et al. Deep representation learning using multilayer perceptron and stacked autoencoder for recommendation systems. in 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 2021.IEEE.Volume 15- Number 4 - 2023 (25 -31) 30 [DOI:10.1109/SMC52423.2021.9658978]
4. [4] Ahmadian, S., et al., A social recommender system based on reliable implicit relationships. Knowledge-Based Systems, 2020. 192: p. 105371. [DOI:10.1016/j.knosys.2019.105371]
5. [5] Mollahosseini, A., D. Chan, and M.H. Mahoor. Going deeper in facial expression recognition using deep neural networks. in 2016 IEEE Winter conference on applications of computer vision (WACV). 2016. IEEE. [DOI:10.1109/WACV.2016.7477450]
6. [6] Tang, Y., Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239, 2013.
7. [7] Xu, M., et al. Facial expression recognition based on transfer learning from deep convolutional networks. in 2015 11th International Conference on Natural Computation (ICNC). 2015. IEEE.
8. [8] Targ, S., D. Almeida, and K. Lyman, Resnet in resnet: Generalizing residual architectures. arXiv preprint arXiv:1603.08029, 2016.
9. [9] Yao, A., et al. HoloNet: towards robust emotion recognition in the wild. in Proceedings of the 18th ACM international conference on multimodal interaction. 2016. [DOI:10.1145/2993148.2997639]
10. [10] Liu, M., et al., Au-inspired deep networks for facial expression feature learning. Neurocomputing, 2015. 159: p. 126-136. [DOI:10.1016/j.neucom.2015.02.011]
11. [11] Rangulov, D. and M. Fahim. Emotion recognition on large video dataset based on convolutional feature extractor and recurrent neural network. in 2020 IEEE 4th International Conference on Image Processing, Applications and Systems (IPAS). 2020. IEEE. [DOI:10.1109/IPAS50080.2020.9334935]
12. [12] Ng, H.-W., et al. Deep learning for emotion recognition on small datasets using transfer learning. in Proceedings of the 2015 ACM on international conference on multimodal interaction. 2015. [DOI:10.1145/2818346.2830593] [PMID]
13. [13] Kaya, H., F. Gürpınar, and A.A. Salah, Video-based emotion recognition in the wild using deep transfer learning and score fusion. Image and Vision Computing, 2017. 65: p. 66-75. [DOI:10.1016/j.imavis.2017.01.012]
14. [14] Khorrami, P., T. Paine, and T. Huang. Do deep neural networks learn facial action units when doing expression recognition? in Proceedings of the IEEE international conference on computer vision workshops. 2015. [DOI:10.1109/ICCVW.2015.12]
15. [15] Kamilaris, A. and F.X. Prenafeta-Boldú, Deep learning in agriculture: A survey. Computers and electronics in agriculture, 2018. 147: p. 70-90. [DOI:10.1016/j.compag.2018.02.016]
16. [16] Chollet, F. Xception: Deep learning with depthwise separable convolutions. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. [DOI:10.1109/CVPR.2017.195]
17. [17] Yengikand, A.K., M. Meghdadi, and S. Ahmadian, DHSIRS: a novel deep hybrid side information-based recommender system. Multimedia Tools and Applications, 2023: p. 1-27. [DOI:10.1007/s11042-023-15021-9]
18. [18] Kingma, D.P. and J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
19. [19] Liu, M., et al. Au-aware deep networks for facial expression recognition. in 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG).2013. IEEE.
20. [20] Samsani, S. and V.A. Gottala. A real-time automatic human facial expression recognition system using deep neural networks. in Information and Communication Technology for Sustainable Development: Proceedings of ICT4SD 2018.2020. Springer. [DOI:10.1007/978-981-13-7166-0_43]
21. [21] Dapogny, A., K. Bailly, and S. Dubuisson, Confidence weighted local expression predictions for occlusion handling in expression recognition and action unit detection. International Journal of Computer Vision, 2018. 126: p. 255-271. [DOI:10.1007/s11263-017-1010-1]
22. [22] Happy, S. and A. Routray, Automatic facial expression recognition using features of salient facial patches. IEEE transactions on Affective Computing, 2014. 6(1): p. 1-12. [DOI:10.1109/TAFFC.2014.2386334]
23. [23] Jung, H., et al. Joint fine-tuning in deep neural networks for facial expression recognition. in Proceedings of the IEEE international conference on computer vision. 2015. [DOI:10.1109/ICCV.2015.341] [PMID]
Send email to the article author
| Rights and permissions | |
 | This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
