高级检索

  • ISSN 1006-3080
  • CN 31-1691/TQ

基于时序卷积网络的情感识别算法

宋振振 陈兰岚 娄晓光

宋振振, 陈兰岚, 娄晓光. 基于时序卷积网络的情感识别算法[J]. 华东理工大学学报(自然科学版), 2020, 46(4): 564-572. doi: 10.14135/j.cnki.1006-3080.20190508001
引用本文: 宋振振, 陈兰岚, 娄晓光. 基于时序卷积网络的情感识别算法[J]. 华东理工大学学报(自然科学版), 2020, 46(4): 564-572. doi: 10.14135/j.cnki.1006-3080.20190508001
SONG Zhenzhen, CHEN Lanlan, LOU Xiaoguang. Emotion Recognition Algorithm Based on Temporal Convolution Network[J]. Journal of East China University of Science and Technology, 2020, 46(4): 564-572. doi: 10.14135/j.cnki.1006-3080.20190508001
Citation: SONG Zhenzhen, CHEN Lanlan, LOU Xiaoguang. Emotion Recognition Algorithm Based on Temporal Convolution Network[J]. Journal of East China University of Science and Technology, 2020, 46(4): 564-572. doi: 10.14135/j.cnki.1006-3080.20190508001

基于时序卷积网络的情感识别算法

doi: 10.14135/j.cnki.1006-3080.20190508001
基金项目: 国家自然科学基金(61201124);中央高校基本科研业务费专项资金(222201917006)
详细信息
    作者简介:

    宋振振(1994-),男,山东菏泽人,硕士生,主要研究方向为情感识别。E-mail:zhenzhensung@163.com

    通讯作者:

    陈兰岚,E-mail:llchen@ecust.edu.cn

  • 中图分类号: TP391

Emotion Recognition Algorithm Based on Temporal Convolution Network

  • 摘要: 采用脑电数据集DEAP进行情感识别。由于脑电信号具有时序性,采用深度学习中的时序卷积网络(TCN)对数据进行训练识别。首先使用小波包分解提取各子带小波系数能量值作为特征;然后通过TCN对特征进行训练,在训练过程中加入了Snapshot寻优思想保存多个模型;最后采用投票集成策略建立集成模型,以提高识别精度,并增强结果稳健性。实验结果表明,本文方法将情感分为二类和四类的平均识别精度分别能够达到95%和93%,相对于同类研究有较大的提高。

     

  • 图  1  基于脑电信号的情感识别算法总体框图

    Figure  1.  Overall block diagram of EEG-based emotion recognition algorithm

    图  2  视频片段得分分布

    Figure  2.  Score distribution of video clip

    图  3  TCN结构图

    Figure  3.  TCN structure diagram

    图  4  因果卷积模型

    Figure  4.  Casual convolution model

    图  5  一维扩张卷积

    Figure  5.  One-dimensional expansion convolution model

    图  6  寻优过程图

    Figure  6.  Diagram of optimization process

    图  7  投票集成过程

    Figure  7.  Process of voting ensemble

    图  8  Valence二分类结果对比

    Figure  8.  Comparison of valence two classification results

    图  9  Arousal二分类结果对比

    Figure  9.  Comparison of arousal two classification results

    图  10  A-V四分类结果对比

    Figure  10.  Comparison of A-V four classification results

    图  11  Valence集成策略结果对比

    Figure  11.  Comparison of valence ensemble strategy results

    表  1  各频段信号的识别精度对比

    Table  1.   Recognition accuracy of signals in each frequency band

    EmotionAccuracy/%All
    DeltaThetaAlphaBeta Gamma
    Valence89.1088.5087.1091.5090.3094.70
    Arousal82.1083.3087.6090.8091.1092.90
    A-V83.7085.108584.9086.5090.70
    下载: 导出CSV

    表  2  两种情绪识别精度对比

    Table  2.   Comparison of two emotion recognition accuracy

    MethodAccuracy/%Category
    ValenceArousal
    Reference[20]85.2080.502
    Reference[21]90.3989.062
    Reference[22]73.3072.502
    Reference[23]81.2181.762
    Reference[24]80.1077.202
    Reference[25]74.1272.062
    TCN94.6792.892
    Proposed method96.4795.722
    下载: 导出CSV

    表  3  4种情绪识别精度对比

    Table  3.   Comparison of four emotion recognition accuracy

    MethodAccuracy/%Category
    Reference[26]70.044
    Reference[27]91.204
    Reference[28]90.204
    TCN90.684
    Proposed method92.494
    下载: 导出CSV
  • [1] ZHANG L, JIANG M, FARID D, et al. Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot[J]. Expert Systems with Applications, 2013, 40(13): 5160-5168. doi: 10.1016/j.eswa.2013.03.016
    [2] DAHL G E, YU D, DENG L, et al. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition[J]. IEEE Transactions on Audio, Speech, and Language Processing, 2012, 20(1): 30-42. doi: 10.1109/TASL.2011.2134090
    [3] HEALEY J, PICARD R. Digital processing of affective signals[C]//Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing. USA: IEEE, 1998: 3749-3752.
    [4] ZHENG W L, LU B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J]. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162-175. doi: 10.1109/TAMD.2015.2431497
    [5] LI X, SONG D, ZHANG P, et al. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network[C]//2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Shenzhen, China: IEEE, 2016: 352-359.
    [6] YANG Y, WU Q M J, ZHENG W L, et al. EEG-based emotion recognition using hierarchical network with subnetwork nodes[J]. IEEE Transactions on Cognitive and Developmental Systems, 2018, 10(2): 408-419. doi: 10.1109/TCDS.2017.2685338
    [7] BENGIO Y. Learning deep architectures for AI[J]. Foundations and Trends® in Machine Learning, 2009, 2(1): 1-127. doi: 10.1561/2200000006
    [8] 李幼军, 黄佳进, 王海渊, 等. 基于SAE和LSTM RNN的多模态生理信号融合和情感识别研究[J]. 通信学报, 2017, 38(12): 109-120. doi: 10.11959/j.issn.1000-436x.2017294
    [9] GREFF K, SRIVASTAVA R K, KOUTNíK J, et al. LSTM: A search space odyssey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2222-2232. doi: 10.1109/TNNLS.2016.2582924
    [10] CHUNG J, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL]. arXiv.org. 2014-12-11[2019-04-20]. arXiv: 1412.3555.
    [11] BAI S, KOLTER J Z, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. arXiv.org. 2018-03-04 [2019-04-20]. arXiv: 1803.01271.
    [12] HUANG G, LI Y, PLEISS G, et al. Snapshot ensembles: Train 1, get m for free[EB/OL].arXiv.org. 2017-04-01[2019-04-20]. arXiv: 1704.00109.
    [13] KOELSTRA S, MUHL C, SOLEYMANI M, et al. DEAP: A database for emotion analysis;using physiological signals[J]. IEEE Transactions on Affective Computing, 2012, 3(1): 18-31. doi: 10.1109/T-AFFC.2011.15
    [14] EGON L, VAN DEN Broek. Ubiquitous emotion-aware computing[J]. Personal and Ubiquitous Computing, 2013, 17(1): 53-67. doi: 10.1007/s00779-011-0479-9
    [15] MEHRABIAN A. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament[J]. Current Psychology, 1996, 14(4): 261-292. doi: 10.1007/BF02686918
    [16] JAMES A R. A circumplex model of affect[J]. Journal of Personality and Social Psychology, 1980, 39(6): 1161-1178. doi: 10.1037/h0077714
    [17] GERS F A, SCHMIDHUBER E. LSTM recurrent networks learn simple context-free and context-sensitive languages[J]. IEEE Transactions on Neural Networks, 2001, 12(6): 1333-1340. doi: 10.1109/72.963769
    [18] OORD A, DIELEMAN S, ZEN H, et al. WaveNet: A generative model for raw audio[EB/OL]. arXiv.org. 2016-09-12[2019-04-20]. arXiv: 1609.03499.
    [19] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. USA: IEEE, 2016: 770-778.
    [20] LIU W, ZHENG W L, LU B L. Multimodal emotion recognition using multimodal deep learning[C]//International Conference on Neural Information Processing. Japan: Springer, 2016: 521-529.
    [21] LIU N, FANG Y, LI L, et al. Multiple feature fusion for automatic emotion recognition using EEG signals[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). USA: IEEE, 2018: 896-900.
    [22] THAMMASAN N, MORIYAMA K, FUKUI K I, et al. Familiarity effects in EEG-based emotion recognition[J]. Brain Informatics, 2017, 4(1): 39-50. doi: 10.1007/s40708-016-0051-5
    [23] ZHANG J, CHEN M, HU S, et al. PNN for EEG-based emotion recognition[C]//2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). USA: IEEE, 2016: 2319-2323.
    [24] KOLLIA V. Personalization effect on emotion recognition from physiological data: An investigation of performance on different setups and classifiers[EB/OL]. arXiv.org. 2016-06-20[2019-04-20]. arXiv: 1607.05832.
    [25] LI X, SONG D, ZHANG P, et al. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network[C]//2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). USA: IEEE, 2016: 352-359.
    [26] CHEN J, HU B, WANG Y, et al. A three-stage decision framework for multi-subject emotion recognition using physiological signals[C]//2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). USA: IEEE, 2016: 470-474.
    [27] VAL-CALVO M, ÁLVAREZ-SÁNCHEZ J R, DÍAZ-MORCILLO A, et al. On the use of lateralization for lightweight and accurate methodology for EEG real time emotion estimation using Gaussian-process classifier[C]//International Work-Conference on the Interplay Between Natural and Artificial Computation. Germany, Cham: Springer, 2019: 191-201.
    [28] AGUIÑAGA A R, RAMIREZ M A L. Emotional states recognition, implementing a low computational com-plexity strategy[J]. Health Informatics Journal, 2018, 24(2): 146-170. doi: 10.1177/1460458216661862
  • 加载中
图(11) / 表(3)
计量
  • 文章访问数:  9557
  • HTML全文浏览量:  4402
  • PDF下载量:  108
  • 被引次数: 0
出版历程
  • 收稿日期:  2019-05-08
  • 网络出版日期:  2019-10-15
  • 刊出日期:  2020-08-01

目录

    /

    返回文章
    返回