Emotion Decoding Model Based on Recurrent Analysis of EEG Functional Connectivity Microstate Sequence
-
摘要: 脑电微状态代表准稳定的全局神经元活动，被认为是大脑动力学的构建模块，而基于微状态概率统计的特征不能很好表征脑电的动态变化特性。针对该问题提出了基于脑电微状态序列递归分析的情绪解码模型。首先，利用通道间的时域相关性建立每帧信号的微状态，并通过聚类获得微状态典型模式。然后，对典型模式时间序列构建递归图表征脑电动力学特性。最后，利用卷积神经网络对递归图进行情绪预测。在DEAP数据集上的实验结果表明该模型取得了比传统微状态方法更好的性能。Abstract: Electroencephalogram (EEG) functional connectivity microstates represent quasi-stable global neuronal activity and are considered the building blocks of brain dynamics. Therefore, microstate sequence analysis is a promising method to understand the brain dynamics behind various emotional states. Recent studies have shown that the sequence of EEG microstates is non-Markov and non-stationary, which also explains the importance of temporal dynamics between different emotional states. However, the microstate features based on probability statistics can not well represent the dynamic characteristics of EEG signals. These findings inspire us to use recurrence analysis to model time series of microstates to capture non-obvious correlations in time series. In conclusion, we propose an emotion decoding model based on recurrence analysis of EEG functional connectivity microstate sequences. Firstly, the functional connection microstate pattern of each frame signal is established by using the correlation of time-domain signals between each channel, and the typical microstate patterns are obtained by clustering. Then, the original EEG signals were mapped to microstate time series according to typical microstate patterns, and the time series were analyzed recursively to construct recurrence plots to characterize the EEG dynamic characteristics. Finally, Convolutional Neural Networks (CNNs) are used to predict the regression of emotions based on the valence or arousal value. On open dataset DEAP, the regression effect of the Mean Square Error (MSE) of the model in the two dimensions of valence and arousal is 3.45±1.42 and 2.79±1.48, respectively, which is better than the MSE of 3.87±1.67 and 3.25±1.71 based on the traditional statistical characteristics of microstate features.
-
Key words:
- brain functional connection /
- microstate /
- recurrence analysis /
- electroencephalogram /
- emotion decoding
-
图 1 基于脑电功能连接微状态序列递归分析的情绪解码模型
Figure 1. Emotion decoding model based on recurrent analysis of functional connectivity microstate sequence of EEG
图 3 微状态典型模式的个数对模型性能的影响
Figure 3. The influence of the number of microstate typical patterns on model performance
图 4
$ K{\text{ = }}5 $ 时基于微状态序列递归分析方法不同频带的性能对比Figure 4. Performance comparison of different frequency bands based on microstate sequence recurrent analysis method when K=5
图 5
$ K{\text{ = }}5 $ 时基于不同频带的微状态典型模式对比结果注：在每个子图中，显示了对应相关系数绝对值最大的 20%的边。红色和蓝色分别表示正关联和负关联，颜色的深浅反映关联权值绝对值的大小。圆的左右部分分别表示大脑的左右区域，左边的电极从上至下按照额区、中央区、颞区、顶区、枕区的位置排列，右边的电极排列相反。
Figure 5. Comparison results are based on typical microstate modes in different frequency bands when K=5
Subject Valence Arousal Ours [9] Ours [9] 1 4.21 6.09 3.94 4.31 2 7.41 7.94 6.40 8.33 3 1.75 1.87 1.62 2.22 4 4.96 4.54 3.04 3.59 5 3.84 4.67 2.34 3.48 6 1.56 1.99 1.77 1.90 7 2.73 3.53 2.99 2.96 8 2.90 3.81 1.77 1.98 9 1.47 1.81 0.82 0.90 10 3.29 3.33 1.69 1.94 11 3.56 3.72 4.11 5.18 12 3.96 4.03 1.88 2.82 13 4.22 4.92 2.38 3.37 14 4.05 4.02 1.80 2.86 15 4.29 3.19 1.49 1.82 16 2.44 3.07 2.49 3.19 17 1.46 1.60 1.50 1.32 18 1.22 1.32 1.31 1.71 19 2.81 3.10 2.39 2.98 20 2.38 2.64 1.24 1.44 21 2.31 3.34 1.34 1.60 22 5.71 5.40 2.99 3.28 23 2.67 2.95 5.20 5.69 24 3.43 4.08 1.49 2.02 25 5.89 6.06 3.63 3.24 26 4.78 6.72 4.34 5.53 27 4.29 4.26 5.61 3.74 28 4.21 6.26 4.71 6.18 29 3.86 4.32 4.87 4.41 30 1.73 1.72 1.65 1.56 31 3.71 3.99 4.36 5.88 32 3.30 3.50 2.12 2.45 Mean 3.45 3.87 2.79 3.25 Std 1.42 1.67 1.48 1.71 
下载: 导出CSV
-
[1] KASEY P, 5 trends appear on the Gartner hype cycle for emerging technologies, 2019[R]. 2019.https://www.gartner.com/smarterwithgartner/5-trends-appear-on-the-gartner-hype-cycle-for-emerging-technologies-2019. [2] SOUJANYA P, ERIK C, RAJIV B, et al. A review of affective computing: From unimodal analysis to multimodal fusion[J]. Information Fusion, 2017, 37: 98-125. doi: 10.1016/j.inffus.2017.02.003 [3] CHRISTIAN M, BRENDAN A, ANTON N, et al. A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges[J]. Brain-Computer Interfaces, 2014, 1(2): 66-84. doi: 10.1080/2326263X.2014.912881 [4] SHANECHI M M. Brain–machine interfaces from motor to mood[J]. Nature neuroscience, 2019, 22(10): 1554-1564. doi: 10.1038/s41593-019-0488-y [5] THANYATHORN T, KATHERINE M, JACQUELINE A. Emotion in a century: A review of emotion recognition[C]//Proceedings of the 10th International Conference on Advances in Information Technology. 2018: 1-8. [6] JOSEPH L. The emotional brain: The mysterious underpinnings of emotional life[M]. Simon and Schuster, 1998. [7] ZHENG W L, ZHU J Y, LV B L. Identifying stable patterns over time for emotion recognition from EEG[J]. IEEE Transactions on Affective Computing, 2017, 10(3): 417-429. [8] GARCIA M. B., MARTINEZ R. A., ALCARAZ R, et al. A review on nonlinear methods using electroencephalographic recordings for emotion recognition[J]. IEEE Transactions on Affective Computing, 2019, 12(3): 801-820. [9] 沈新科, 李奕超, 刘锦, 等. 基于脑电功能连接微状态的情绪状态解码[J]. 智能科学与技术学报, 2021, 3(01): 49-58. [10] SHI L C, JIAO Y Y, LV B L. Differential entropy feature for EEG-based vigilance estimation[C]//2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2013: 6627-6630. [11] PETRANTONAKIS P C, HADJILEONTIADIS L J. Emotion recognition from EEG using higher order crossings[J]. IEEE Transactions on information Technology in Biomedicine, 2009, 14(2): 186-197. [12] WU X, ZHENG W L, LV B L. Investigating EEG-based functional connectivity patterns for multimodal emotion recognition[J]. arXiv preprint arXiv: 2004.01973, 2020. [13] LI P Y, LIU H A, SI Y J, et al. EEG based emotion recognition by combining functional connectivity network and local activations[J]. IEEE Transactions on Biomedical Engineering, 2019, 66(10): 2869-2881. doi: 10.1109/TBME.2019.2897651 [14] SONG T F, ZHENG W M, SONG P, et al. EEG emotion recognition using dynamical graph convolutional neural networks[J]. IEEE Transactions on Affective Computing, 2018, 11(3): 532-541. [15] O'NEILL G C, TEWARIE P, VIDAURRE D, et al. Dynamics of large-scale electrophysiological networks: A technical review[J]. Neuroimage, 2018, 180: 559-576. doi: 10.1016/j.neuroimage.2017.10.003 [16] APOORVA S, HAMIDREZA J, MARINA K. Investigating the temporal dynamics of electroencephalogram (EEG) microstates using recurrent neural networks[J]. Human brain mapping, 2020, 41(9): 2334-2346. doi: 10.1002/hbm.24949 [17] ARTHUR D, VASSILVITSKII S. k-means++: The advantages of careful seeding[R]. Stanford, 2006. [18] NORBERT M, M CARMEN R, MARCO T, et al. Recurrence plots for the analysis of complex systems[J]. Physics reports, 2007, 438(5-6): 237-329. doi: 10.1016/j.physrep.2006.11.001 [19] NIMA H, YANN G, JOHAN D. Classification of time-series images using deep convolutional neural networks[C]//Tenth international conference on machine vision (ICMV 2017). International Society for Optics and Photonics, 2018, 10696: 106960Y. [20] KOELSTRA S, MUHL C, SOLEYMANI M, et al. Deap: A database for emotion analysis; using physiological signals[J]. IEEE transactions on affective computing, 2011, 3(1): 18-31. [21] ARNAUD D, SCOTT M. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis[J]. Journal of neuroscience methods, 2004, 134(1): 9-21. doi: 10.1016/j.jneumeth.2003.10.009 [22] LYUDMYLA K, TAMARA R, JULIIA S. Applying recurrence plots to classify time series[J]. COMPUTATIONAL LINGUISTICS AND INTELLIGENT SYSTEMS, 2021, 2: 16-26. -
点击查看大图
图(5) / 表(1)
计量
- 文章访问数: 92
- HTML全文浏览量: 33
- PDF下载量: 6
- 被引次数: 0
