北京生物医学工程

脑电与功能近红外光谱技术在脑机接口中的应用

Applications of EEG and fNIRS in brain computer interface

作者：高宇航司娟宁何江弘李梦

单位：北京信息科技大学仪器科学与光电工程学院（北京 100192） 解放军总医院第七医学中心神经外科（北京 100700） 通信作者：司娟宁。E-mail:  sijuanning@bistu.edu.cn

关键词：脑-机接口；人机交互；脑电；功能近红外光谱；多模态信息融合

分类号：R318.04 

出版年·卷·期（页码）：2022·41·3（318-326）

摘要：

脑-机接口(brain-computer interface, BCI)技术是一种多学科交叉融合的新型人机交互方式，通过解码大脑的活动信息来控制外部设备，从而实现人脑与外界的信息交互，在神经科学、康复医疗、人工智能等领域应用广泛。近年来随着科技进步，多尺度（宏观、介观、微观）脑成像技术不断涌现，如脑电图（electroencephalogram，EEG)、功能磁共振成像（functional magnetic resonance imaging，fMRI)、功能近红外光谱（functional near-infrared spectroscopy，fNIRS)，极大地推动了BCI的发展。本文综述了EEG、fNIRS及EEG-fNIRS多模态融合技术在BCI中的应用现状，归纳各技术的研究成果，探讨其局限性和改进方式，并对未来BCI的发展做了展望。

Brain-computer interface (BCI) technology is a new type of human-computer interaction technique with interdisciplinary integration, which controls external devices by decoding brain activity information, thereby realizing information interaction between the human brain and the outside world. BCI is widely used in neuroscience, rehabilitation medicine, artificial intelligence and other fields. In recent years, with the progress of neuroimaging technology, multi-scale (macro, mesoscopic and micro) brain imaging technologies，such as electroencephalogram (EEG), functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS), have been constantly emerging, and have greatly promoted the development of BCI. This paper reviews the application status of EEG, fNIRS and EEG-fNIRS multimodal fusion technology in BCI, summarizes the research achievements of each technology and discusses their limitations and improvement methods. Finally, the future development direction of BCI research is prospected.

参考文献：

[1] Vidal J J. Real-time detection of brain events in EEG[J]. Proceedings of the IEEE, 1977, 65(5): 633-641.

[2] Lazeyras F O, Blanke O, Perrig S, et al. EEG-triggered functional MRI in patients with pharmacoresistant epilepsy[J]. Journal of Magnetic Resonance Imaging, 2015, 12(1): 177-185.

[3] 黄帅, 杜云梅, 梁会营, 等. 基于EEG的癫痫发作预测当前研究与进展[J]. 中国数字医学, 2019, 14(3):73-76.

Huang S, Du YH, Liang HY, et al. Current research and development of epileptic seizure prediction based on EEG [J]. 中国数字医学, 2019, 14(3):73-76.

[4] 吴迪, 王红新, 郝小生, 等. 剥夺睡眠EEG与脑电Holter对伴中央-颞区棘波的儿童良性癫痫诊断应用比较[J]. 吉林医学, 2012, 33(13):2744-2745.

Wu D, Wang HX, Hao XS, et al. Comparison of sleep deprivation EEG and EEG Holter in the diagnosis of benign epilepsy in children with center-temporal spines[J]. Jilin medical, 2012, 33(13):2744-2745.

[5] Mishuhina V, Jiang X. Feature Weighting and Regularization of Common Spatial Patterns in EEG-Based Motor Imagery BCI[J]. IEEE Signal Processing Letters, 2018, 25(6):783-787.

[6] 许学添, 蔡跃新. 基于图卷积网络的运动想象识别[J/OL]. 计算机工程与应用, http://kns.cnki.net/kcms/detail/11.2127. TP.20210119.1330.014.html, 2021-3-5.

Xu XT, Cai YX. Motion Imagination Recognition Based on Graph Convolutional Network [J/OL]. Computer Engineering and? Applications, http://kns.cnki.net/kcms/detail/11.2127.TP.20210119.1330.014.html, 2021-3-5.

[7] Holz E M, Botrel L, Kaufmann T, et al. Long-Term Independent Brain-Computer Interface Home Use Improves Quality of Life of a Patient in the Locked-In State: A Case Study[J]. Arch Phys Med Rehabil, 2015, 96(3): S16-S26.

[8] Boonyakitanont P, Lek-Uthai A, Chomtho K, et al. A review of feature extraction and performance evaluation in epileptic seizure detection using EEG[J]. Biomedical Signal Processing and Control, 2020, 57(4): 1746-1762.

[9] 张锦, 刘熔, 田森, 等. 面向癫痫脑电的简化深度学习模型[J]. 国防科技大学学报, 2020, 42(6): 106-111.

Zhang J, Liu R, Tian S, et al. A simplified deep learning model for epileptic EEG [J]. Journal of National University of Defense Technology, 2020, 42(6): 106-111.

[10] Gaur P, Pachori R B, Wang H, et al. A multi-class EEG-based BCI classification using multivariate empirical mode decomposition based filtering and Riemannian geometry[J]. Expert Systems with Applications, 2018, 95(APR.): 201-211.

[11] 付荣荣, 田永胜, 鲍甜恬. 基于稀疏共空间模式和Fisher判别的单次运动想象脑电信号识别方法[J]. 生物医学工程学杂志, 2019, 36(6): 911-915，923.

Fu RR, Tian YS, Bao TT. A Single Motor Imagination EEG Signal Recognition Method Based on Sparse Common Space Pattern and Fisher Discriminant[J]. Journal of Biomedical Engineering, 2019, 36(6): 911-915，923.

[12] Pan J, Xie Q, Huang H, et al. Emotion-Related Consciousness Detection in Patients With Disorders of Consciousness Through an EEG-Based BCI System[J]. Front Hum Neurosci, 2018, 12(6):198-209.

[13] Talukdar U, Hazarika S M, Gan J Q. Adaptive feature extraction in EEG-based motor imagery BCI: tracking mental fatigue[J]. Journal of neural engineering, 2020, 17(1): 016020.1-016020.16.

[14] Sakurada T, Hirai M, Watanabe E. Individual optimal attentional strategy during implicit motor learning boosts frontoparietal neural processing efficiency: A functional near-infrared spectroscopy study[J]. Brain Behav, 2019, 9(1): e01183.

[15] Li C, Su M, Xu J, et al. A Between-Subject fNIRS-BCI Study on Detecting Self-Regulated Intention During Walking[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(2): 531-540.

[16] 李玉, 熊馨, 李昭阳,等. 基于功能性近红外光谱识别右脚三种想象动作研究[J]. 生物医学工程学杂志, 2020, 37(2):262-270.

Li Y, Xiong X, Li ZY, et al. Study on Recognition of Three Imaginary Movements of Right Foot Based on Functional Near Infrared Spectroscopy[J]. Journal of Biomedical Engineering, 2020, 37(2):262-270.

[17] Arivudaiyanambi J, Mohan S, Chhabra H, et al. Investigation of deep convolutional neural network for classification of motor imagery fNIRS signals for BCI applications[J]. Biomedical Signal Processing and Control, 2020, 62(4):102-133.

[18] Chaudhary U, Xia B, Silvoni S, et al. Correction: Brain-Computer Interface-Based Communication in the Completely Locked-In State[J]. Plos Biology, 2018, 16(12): e3000089.

[19] Abdalmalak A, Milej D, Norton L, et al. Single-session communication with a locked-in patient by functional near-infrared spectroscopy[J]. Neurophotonics, 2017, 4(4):040501.

[20] 阮盛捷, 谢宏. 基于功能近红外信号的脑机接口(BCI)在肌萎缩性侧索硬化症(ALS)患者交流中的研究[J]. 现代计算机(专业版), 2018, 617(17): 5-9.

Ruan SJ, Xie H. Brain computer interface (BCI) based on functional near-infrared signals in the communication of patients with amyotrophic lateral sclerosis (ALS)[J]. Modern Computer (Professional), 2018, 617(17): 5-9.

[21] Borgheai S B, Mclinden J, Zisk A H, et al. Enhancing Communication for People in Late-Stage ALS Using an fNIRS-Based BCI System[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(5): 1198-1207.

[22] Pinti P, Tachtsidis I, Hamilton A, et al. The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience[J]. Ann N Y Acad Sci, 2020, 1464(1): 5-29.

[23] J?bsis F F. Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters [J]. Science, 1977, 198(4323): 1264-1267.

[24] Li C, Zhang M, Ding K, et al. Effect of English learning experience on young children's prefrontal cortex functioning for attentional control: An fNIRS study.[C]// 2019 41st Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2019, 2019:4832-4835.

[25] 郑玉甫, 许敏鹏, 明东. 脑-机接口操控效果差异及其预测研究综述[J]. 中国生物医学工程学报, 2018, 37(6): 749-755.

Zheng YF, Xu MP, Ming D. A review of research on differences and prediction of brain-computer interface manipulation effects[J]. Chinese Journal of Biomedical Engineering, 2018, 37(6): 749-755.

[26] Verma P , Heilinger A , Reitner P , et al. Performance Investigation of Brain-Computer Interfaces that Combine EEG and fNIRS for Motor Imagery Tasks[C]// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE, 2019, 259-263.

[27] Sargent A , Heiman-Patterson T , Feldman S , et al. Mental Fatigue Assessment in Prolonged BCI Use Through EEG and fNIRS[J]. Neuroergonomics, 2018, 5(3):315-316.

[28] Nguyen T, Ahn S, Jang H, et al. Utilization of a combined EEG/NIRS system to predict driver drowsiness[J]. Entific Reports, 2017, 7(1):43933.

[29] 熊馨, 伏云发, 张夏冰, 等. 一种多模态脑电和近红外光谱联合采集头盔设计及实验研究[J]. 生物医学工程学杂志, 2018, 35(2): 290-296.

Xiong X, Fu YF, Zhang XB, et al. Design and Experimental Study of a Helmet Collected by Multi-modal EEG and Near Infrared Spectra[J]. Journal of Biomedical Engineering, 2018, 35(2): 290-296.

[30] Chiarelli A M, Croce P, Merla A, et al. Deep Learning for hybrid EEG-fNIRS Brain-Computer Interface: application to Motor Imagery Classification[J]. Journal of Neural Engineering, 2018, 15(3): 036028.1-036028.12.

[31] Dehais F , Dupres A , Flumeri G D , et al. Monitoring Pilot's Cognitive Fatigue with Engagement Features in Simulated and Actual Flight Conditions Using an Hybrid fNIRS-EEG Passive BCI[C]// 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2018.

[32] Sereshkeh A R, Yousefi R, Wong A T, et al. Development of a ternary hybrid fNIRS-EEG brain–computer interface based on imagined speech[J]. Brain-Computer Interfaces, 2019, 6(2):1-13.

[33] Gentile E, Brunetti A, Ricci K, et al. Mutual interaction between motor cortex activation and pain in fibromyalgia: EEG-fNIRS study[J]. PLoS ONE, 2020, 15(1): e0228158.

[34] Morioka H, Kanemura A, Morimoto S, et al. Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information [J]. Neuroimage, 2014, 90:128-139.

[35] Sangtae A, Thien N, Hyojung J, et al. Exploring Neuro-Physiological Correlates of Drivers' Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data[J]. Frontiers in Human Neuroence, 2016, 10(848):219-233.

服务与反馈：

【文章下载】【加入收藏】

提示：您还未登录，请登录！点此登录