北京生物医学工程

基于脑电信号的情感识别方法综述

A survey of emotion recognition method based on EEG signals

作者：孙中皋薛全德王新军黄晓理

单位：辽宁师范大学物理与电子技术学院（辽宁大连 116029）

关键词：情感识别; 综述; 脑电; 特征提取; 机器学习

分类号：R318；TP391.4

出版年·卷·期（页码）：2020·39·2（186-195）

摘要：

情感识别是让计算机感知人类情感状态从而进行人机情感交互的关键技术，已经成为人工智能领域的研究热点。计算机对人类情感的感知可分为“感”和“知”两部分：“感”是指计算机对人类面部表情和语音等非生理信号以及外围神经和脑部电信号等生理信号的获取；“知”是指计算机对获取信号的识别和推断。基于脑电信号的感知方法因其具有较高的识别率而成为情感识别最主要的手段之一，其主要步骤为脑电信号获取、预处理、特征提取以及分类识别。本文对脑电情感识别方法中各步骤所涉及的研究方法进行了归纳和总结，介绍了脑电信号的采集和常用数据库以及去除伪迹信号的预处理方法，从时域、频域、时频域和非线性动力学角度归纳了脑电信号的特征提取方法，总结了常用的机器识别分类的无监督学习和有监督学习方法，最后探讨了脑电信号情感识别研究中存在的问题并展望了未来发展方向，以期为相关研究带来一定的借鉴。

Emotion recognition is a key technology that allows computers to perceive the emotional state of human beings and thus engage in human-computer emotional interaction. It has become a research hotspot in the field of artificial intelligence. The perception of human emotions by computers can be divided into two parts: “sense” and “knowledge”. “Sense” refers to the acquisition of physiological signals such as human facial expressions and speech and other physiological signals such as peripheral nerves and brain electrical signals; "Knowledge" refers to the recognition and inference. Emotional recognition based on Electroencephalogram (EEG) has become the main research method because of its high recognition rate and its main steps are EEG acquisition, preprocessing, feature extraction and classification recognition. In this paper, the research methods involved in each step of EEG emotion recognition are summarized. The collection of EEG signals, common databases and preprocessing methods of removing artifacts are introduced. The feature extraction methods of EEG are summarized from the perspectives of time domain, frequency domain, time-frequency domain and nonlinear dynamics and the unsupervised learning and supervised learning methods of machine recognition classification are introduced. Finally, the paper discusses the existing problems in the research of emotion recognition of EEG signals and looks forward to the future development direction, in order to bring some reference for the related research.

参考文献：

[1] Sharon YT, Safiya UN. Emotions, technology, and design[M]. American: Academic Press, 2016.

[2] Mühl C, Allison B, Nijholt A, et al. A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges[J]. Brain-Computer Interfaces, 2014, 1(2): 66-84.

[3] James W. What is an emotion[J]. Mind, 1884, 9(34): 188-205.

[4] Lange CG. The Emotions: A psychophysiological study[J]. The Emotions, 1885: 33-90.

[5] Cannon WB. The James-Lange theory of emotions: a critical examination and an alternative theory[J]. The American Journal of Psychology, 1927, 39(1/4): 106-124.

[6] Papez JW. A proposed mechanism of emotion[J]. Archives of Neurology and Psychiatry, 1937, 38(4): 725-743.

[7] Maclean PD. Psychosomatic disease and the “Visceral Brain” recent developments bearing on the papez theory of emotion[J]. Psychosomatic Medicine, 1949, 11(6): 338-353.

[8] Yin Z, Zhao M, Wang Y, et al. Recognition of emotions using multimodal physiological signals and an ensemble deep learning model[J]. Computer Methods and Programs in Biomedicine, 2017, 140: 93-110.

[9] 贲晛烨, 杨明强, 张鹏, 等. 微表情自动识别综述[J]. 计算机辅助设计与图形学学报, 2014, 26(9): 1385-1395.

Ben XY, Yang MQ, Zhang P, et al. Survey on automatic micro expression recognition methods[J]. Journal of Computer-Aided Design & Computer Graphics, 2014, 26(9): 1385-1395.

[10] Kaur B, Singh D, Roy PP. EEG based emotion classification mechanism in BCI[J]. Procedia Computer Science, 2018, 132: 752-758.

[11] Anderson K, McOwan PW. A real-time automated system for the recognition of human facial expressions[J]. IEEE Transactions on System, Man and Cybernetics, Part B: Cybernetics, 2006, 36(1): 96-105.

[12] 缪裕青，邹巍，刘同来，等. 基于参数迁移和卷积循环神经网络的语音情感识别[J]. 计算机工程与应用, 2019, 55(10): 135-140, 198.

Miao YQ, Zou W, Liu TL, et al. Speech emotion recognition model based on parameter transfer and convolutional recurrent neural network[J]. Computer Engineering and Applications, 2019, 55(10): 135-140, 198.

[13] Picard RW, Vyzas E, Healey J. Toward machine emotional intelligence: analysis of affective physiological state[J]. IEEE Transactions on Pattern Analysis and Machined Intelligence, 2001, 23(10): 1175-1191.

[14] Jie X, Cao R, Li L. Emotion recognition based on the sample entropy of EEG[J]. Bio-Medical Materials and Engineering, 2014, 24(1): 1185-1192.

[15] Kaur B, Singh D, Roy PP. A novel framework of EEG-based user identification by analyzing music-listening behavior[J]. Multimedia Tools and Applications, 2017, 76(24): 25581-25602.

[16] Saini R, Kaur B, Singh P, et al. Don’t just sign use brain too: A novel multimodal approach for user identification and verification[J]. Information Sciences, 2018, 430-431: 163-178.

[17] Koelstra S, Mühl C, Soleymani M, et al. DEAP: A database for emotion analysis using physiological signals[J]. IEEE Transactions on Affective Computing, 2012, 3(1): 18-31.

[18] Soleymani M, Lichtenauer J, Pun T, et al. A multimodal database for affect recognition and implicit tagging[J]. IEEE Transactions on Affective Computing, 2012, 3(1): 42-55.

[19] Zheng WL, Lu BL. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J]. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162–175.

[20] Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): Instruction manual and affective ratings, Technical Report A-8. Gainesville: The Center for Research in Psychophysiology, University of Florida, 2008.

[21] Bradley MM, Lang PJ. International affective digitized sounds (IADS): Stimuli, instruction manual and affective ratings. Technical report B-2. Gainesville, FL: The Center for Research in Psychophysiology, University of Florida, 1999.

[22] Dai YX, Wang X, Zhang PB, et al. Sparsity constrained differential evolution enabled feature-channel-sample hybrid selection for daily-life EEG emotion recognition[J]. Multimedia Tools and Applications, 2018, 77(17): 21967-21994.

[23] Yu XY, Chum P, Sim KB. Analysis the effect of PCA for feature reduction in non-stationary EEG based motor imagery of BCI system[J]. Optik-International Journal for Light and Electron Optics, 2014, 125(3): 1498-1502.

[24] Artoni F, Delorme A, Makeig S. Applying dimension reduction to EEG data by principal component analysis reduces the quality of its subsequent independent component decomposition[J]. NeuroImage, 2018, 175: 176-187.

[25] Liang Z, Oba S, Ishii S. An unsupervised EEG decoding system for human emotion recognition[J]. Neural Networks, 2019, 116: 257-268.

[26] Zhang Q, Lee M. A hierarchical positive and negative emotion understanding system based on integrated analysis of visual and brain signals[J]. Neurocomputing, 2010, 73(16-18): 3264-3272.

[27] 聂聃, 王晓鞲, 段若男, 等. 基于脑电的情绪识别研究综述[J]. 中国生物医学工程学报, 2012, 31(4): 595-606.

Nie D, Wang XW, Duan RN, et al. A survey on EEG based emotion recognition[J]. Chinese Journal of Biomedical Engineering, 2012, 31(4): 595-606.

[28] Zouridakis G, Patidar U, Padhyen NS, et al. Spectral power of brain activity associated with emotion—a pilot MEG study[C]// Proceedings of the 17th International Conference on Bio-magnetism Advances in Biomagnetism-BIOMAG2010, IFMBE Proceedings. Dubrovnik, Croatia: Springer, 2010, 28(1): 354-357.

[29] Xing BX, Zhang H, Zhang K, et al. Exploiting EEG signals and audiovisual feature fusion for video emotion recognition[J]. IEEE Access, 2019, 7: 59844-59861.

[30] Wu N, Jiang H, Yang G. Emotion recognition based on physiological signals[J]. Advances in Brain Inspired Cognitive Systems, 2012, 7366: 311-320.

[31] 裴一飞, 杨淑娟.运动想象脑电信号算法研究进展[J]. 北京生物医学工程, 2018, 37(2): 208-214.

Pei YF, Yang SJ. Research progress on motor imagery EEG signals[J]. Beijing Biomedical Engineering, 2018, 37(2): 208-214.

[32] 张迪, 万柏坤, 明东. 基于生理信号的情绪识别研究进展[J]. 生物医学工程学杂志, 2015, 32(1): 229-234.

Zhang D, Wan BK, Ming D. Research progress on emotion recognition based on physiological signals[J].Journal of Biomedical Engineering, 2015, 32(1): 229-234.

[33] 李立, 曹锐, 相洁. 脑电数据近似熵与样本熵特征对比研究[J]. 计算机工程与设计, 2014, 35( 3): 1021-1026.

Li L, Cao R, Xiang J. Comparative study of approximate entropy and sample entropy based on characterization of EEG[J]. Computer Engineering and Design, 2014, 35(3): 1021-1026.

[34] Konstantinidis EI, Frantzidis CA, Pappas C, et al. Real time emotion aware applications: a case study employing emotion evocative pictures and neuro-physiological sensing enhanced by graphic processor units[J]. Computer Methods and Programs in Biomedicine, 2012, 107(1): 16-27.

[35] Liu Y, Sourina O, Nguyen MK. Real-time EEG-based emotion recognition and its applications[J]. Transactions on Computational Science XⅡ, 2011, 6670: 256-277.

[36] 张冠华, 余旻婧, 陈果,等. 面向情绪识别的脑电特征研究综述[J]. 中国科学:信息科学, 2019, 49(9): 1097-1118.

Zhang GH, Yu MJ, Chen G, et al. A review of EEG features for emotion recognition[J]. Scientia Sinica Informationis, 2019, 49(9): 1097-1118.

[37] 王亚真, 张新峰, 胡广芹,等. 基于支持向量机的中医舌图像质量评价研究[J].北京生物医学工程, 2015, 34(6): 551-557.

Wang YZ, Zhang XF, Hu GQ, et al. Study of tongue image quality assessment based on SVM in traditional Chinese medicine[J]. Beijing Biomedical Engineering, 2015, 34(6): 551-557.

[38] Liu YJ, Yu MJ, Zhao GZ, et al. Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Transactions on affective Computing, 2017, 99: 1-14.

[39] Nie D, Wang XW, Shi LC, et al. EEG-based emotion recognition during watching movies[C]//5th International Conference on Neural Engineering. Cancun: IEEE Press, 2011: 667-670.

[40] Estepp JR, Klosterman SL, Christensen JC. An assessment of non-stationarity in physiological cognitive state assessment using artificial neural networks[C]//Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society. Boston, Massachusetts: IEEE Press, 2011: 6552-6555.

[41] 陈继华, 李岚, 钱坤喜. 基于多生理信号的情绪初步识别[J]. 生物医学工程研究, 2006, 25(3): 141-146.

Chen JH, Li L, Qian KX. Emotion recognition by physiological signals[J]. Journal of Biomedical Engineering Research, 2006, 25(3): 141-146.

[42] Tipping ME. Sparse Bayesian learning and the relevance vector machine[J]. Journal of Machine Learning Research, 2001, 1:211-244.

[43] 李志文, 蔡先发, 韦佳, 等. 基于PCA和LDA方法的肿瘤基因表达谱数据分类[J].北京生物医学工程, 2014, 33(1): 47-51.

Li ZW, Cai XF, Wei J, et al. Classification of cancer gene expression profile based on PCA and LDA[J]. Beijing Biomedical Engineering, 2014, 33(1): 47-51.

[44] Agrafioti F, Hatzinakos D, Anderson A. ECG pattern analysis for emotion detection[J]. IEEE Transactions on Affective Computing, 2012, 3(1): 102-115.

[45] Kaur B, Singh D, Roy PP. A novel framework of EEG-based user identification by analyzing music-listening behavior[J]. Multimedia Tools and Applications, 2017, 76(24): 25581-25602.

[46] Zhang Y, Ji XM, Zhang SH. An approach to EEG-based emotion recognition using combined feature extraction method[J]. Neuroscience Letters, 2016, 633: 152-157.

[47] 魏琛, 陈兰岚, 张傲. 基于集成卷积神经网络的脑电情感识别[J]. 华东理工大学学报(自然科学版), 2019, 45(4): 614-622.

Wei C, Chen LL, Zhang A. Emotion recognition of EEG based on ensemble convolutional neural networks[J]. Journal of East China University of Science and Technology (Natural Science Edition), 2019, 45(4): 614-622.

[48] Soleymani M, Lichtenauer J, Pun T, et al. A multimodal database for affect recognition and implicit tagging[J]. IEEE Transactions on Affective Computing, 2012, 3(1):42-55.

[49] Taran S, Bajaj V. Emotion recognition from single-channel EEG signals using a two-stage correlation and instantaneous frequency-based filtering method[J]. Computer Methods and Programs in Biomedicine, 2019, 173: 157-165.

[50] AbdelAal MA, Alsawy AA, Hefny HA. EEG-based emotion recognition using a wrapper-based feature selection method[C] // International Conference on Advanced Intelligent Systems and Informatics. Cairo, Egypt: AISI, 2018, 639: 247-256.

[51] Gupta A, Sahu H, Nanecha N, et al. Enhancing text using emotion detected from EEG signals[J]. Journal of Grid Computing, 2019,17(2): 324-340.

[52] Vijayan AE. EEG-based emotion recognition using statistical measures and auto-regressive modeling[C]// IEEE International Conference on Computational Intelligence & Communication Technology (CICT 2015). Ghaziabad, India：IEEE Press, 2015: 587-591.

[53] Mehmood RM, Lee HJ. Towards emotion recognition of EEG brain signals using Hjorth parameters and SVM[J]. Advanced Science and Technology Letters, 2015, 91: 24-27.

[54] Gupta V, Chopda MD, Pachori RB. Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals[J]. IEEE Sensors Journal, 2019, 19(6): 2266-2274.

[55] Chen JX, Zhang PW, Mao ZJ, et al. Accurate EEG-based emotion recognition on combined features using deep convolutional neural networks[J]. IEEE Access, 2019, 7: 44317-44328.

[56] Zhang T, Zheng W, Cui Z, et al. Spatial-temporal recurrent neural network for emotion recognition[J]. IEEE Transactions on Cybernetics, 2018, 49(3): 839-847.

服务与反馈：

【文章下载】【加入收藏】

提示：您还未登录，请登录！点此登录