高级检索

  • ISSN 1006-3080
  • CN 31-1691/TQ

基于时钟触发长短期记忆的多元时序预测

冯勇 冯述放 罗娜

冯勇, 冯述放, 罗娜. 基于时钟触发长短期记忆的多元时序预测[J]. 华东理工大学学报(自然科学版). doi: 10.14135/j.cnki.1006-3080.20211102001
引用本文: 冯勇, 冯述放, 罗娜. 基于时钟触发长短期记忆的多元时序预测[J]. 华东理工大学学报(自然科学版). doi: 10.14135/j.cnki.1006-3080.20211102001
FENG Yong, FENG Shufang, LUO Na. Multivariate Time Series Prediction Based on Clockwork Triggered Long Short Term Memory[J]. Journal of East China University of Science and Technology. doi: 10.14135/j.cnki.1006-3080.20211102001
Citation: FENG Yong, FENG Shufang, LUO Na. Multivariate Time Series Prediction Based on Clockwork Triggered Long Short Term Memory[J]. Journal of East China University of Science and Technology. doi: 10.14135/j.cnki.1006-3080.20211102001

基于时钟触发长短期记忆的多元时序预测

doi: 10.14135/j.cnki.1006-3080.20211102001
详细信息
    作者简介:

    冯勇:作者简介:冯 勇(1997—),男,安徽人,硕士生,主要研究方向为时间序列预测。E-mail:2537138965@qq.com

    通讯作者:

    罗 娜, E-mail:naluo@ecust.edu.cn

  • 中图分类号: TP183

Multivariate Time Series Prediction Based on Clockwork Triggered Long Short Term Memory

  • 摘要: 在现有的多元时间序列预测方法中,模型无法敏锐地捕获时间序列短期突变信号从而导致预测趋势滞后和误差较大。本文提出了一种基于时钟触发长短期记忆 (Clockwork Triggered Long Short Term Memory, CWTLSTM) 网络的多元时序预测模型,通过增强对短期信息的捕获能力提高了预测精度。CWTLSTM将网络中所有的神经元进行分组,对每个分组赋予不同的激活频率,每一组神经元只在时间步长等于周期的整数倍时才被激活。根据周期是否为1将网络分为主干网络链和短期输入增强链,短期输入增强链在靠近输出位置的时间步上激活时,将输入信息的运算结果单向地传递给主干网络链,增强此时的输入权重,使模型在存储长期信息的基础上能快速响应短期突变信息带来的数据波动。在空气污染数据集和水泥篦冷机数据集上的验证结果表明,本文模型在减少预测误差与趋势判断上均有较好的表现。

     

  • 图  1  CWTLSTM结构图

    Figure  1.  Structure of CWTLSTM

    图  2  分频规则为[1,2,1]的CWTLSTM

    Figure  2.  CWTLSTM with frequency division rule [1,2,1]

    图  3  短期信息增益说明

    Figure  3.  Description of short-term information gain

    图  4  一氧化碳浓度的原始数据曲线与ACF曲线

    Figure  4.  Original data curves and ACF curves of carbon monoxide concentration

    图  5  空气污染结果对比 (Timesteps =48)

    Figure  5.  Comparison of air pollution results(Timesteps =48)

    图  6  二次风温度原始数据曲线与ACF曲线

    Figure  6.  Original data curves and ACF curves of secondary air temperature

    图  7  二次风温度预测结果对比(Timesteps=50)

    Figure  7.  Comparison of prediction results of secondary air temperature(Timesteps=50)

    图  8  不同时间步长的结果比对

    Figure  8.  Comparison of results of different time step lengths

    图  9  不同批次结果比对

    Figure  9.  Comparison of results of different batches

    图  10  多组测试的RMSE对比

    Figure  10.  RMSE comparison for multiple tests

    图  11  不同分组频率结果对比

    Figure  11.  Comparison of results of different grouping frequencies

    表  1  空气污染数据集上各模型的预测结果对比

    Table  1.   Comparison of prediction results of various models on the air pollution dataset

    ModelTimesteps=10Timesteps=48
    RMSE/
    ($\mathrm{m}\mathrm{g}\cdot {\mathrm{m} }^{3}$)
    MAE/
    ($\mathrm{m}\mathrm{g}\cdot{\mathrm{m} }^{3}$)
    MAPE/
    %
    ${{\boldsymbol{R}}} ^{2}$RMSE/
    ($\mathrm{m}\mathrm{g}\cdot{\mathrm{m} }^{3}$)
    MAE/
    ($\mathrm{m}\mathrm{g}\cdot{\mathrm{m} }^{3}$)
    MAPE/
    %
    ${{\boldsymbol{R}}} ^{2}$
    VARMAX0.5720.3770.2500.8410.5870.3910.2410.833
    SVR0.5110.3420.2150.8730.5150.3450.1960.872
    XGBoost0.5070.3500.2090.8750.5750.3590.1920.840
    CWRNN0.4850.3430.2280.8860.4980.3810.2230.880
    LSTM0.4830.3300.2150.8870.4650.3220.2210.895
    CWTLSTM0.4780.3220.2050.8890.4250.3020.2160.912
    下载: 导出CSV

    表  2  二次风温度预测结果对比

    Table  2.   Comparison of prediction results of secondary air temperature

    ModelTimesteps=20Timesteps=50
    RMSE
    /$ \mathrm{℃} $
    MAE
    /$ \mathrm{℃} $
    MAPE
    /%
    $ {\mathrm{R}}^{2} $RMSE/
    ($ \mathrm{℃} $)
    MAE/
    ($ \mathrm{℃} $)
    MAPE/
    %
    $ {\mathrm{R}}^{2} $
    VARMAX4.243.390.3120.9423.873.060.2820.952
    SVR3.813.160.2910.9534.373.520.3250.938
    XGBoost4.253.210.2970.9414.243.220.2960.942
    LSTM2.501.930.1760.9732.722.290.2110.976
    CWRNN2.321.890.1740.9822.301.800.1640.983
    CWTLSTM2.231.710.1560.9831.671.330.1220.991
    下载: 导出CSV
  • [1] 任伟杰, 韩敏. 多元时间序列因果关系分析研究综述[J]. 自动化学报, 2021, 47(1): 64-78.
    [2] 辛治运, 顾明, 等. 基于最小二乘支持向量机的复杂金融时间序列预测[J]. 清华大学学报(自然科学版), 2008, 48(7): 82-84.
    [3] KIM J, SIM A. A new approach to multivariate network traffic analysis[J]. Journal of Computer Science and Technology, 2019, 34(2): 388-402. doi: 10.1007/s11390-019-1915-y
    [4] 蔡涛, 杨博, 李宏光. 基于时延挖掘模糊时间认知图的化工过程多变量时序预测方法[J]. 化工学报, 2020, 71(3): 1095-1102.
    [5] SHANG B, SHANG P. Directed vector visibility graph from multivariate time series: A new method to measure time series irreversibility[J]. Nonlinear Dynamics, 2021, 104(2): 1737-1751. doi: 10.1007/s11071-021-06340-3
    [6] SINGHAL A, SEBORG D E. Clustering multivariate time-series data[J]. Journal of Chemometrics, 2010, 19(8): 427-438.
    [7] TiAO G C, TSAY R S. Model specification in multivariate time series[J]. Journal of the Royal Statistical Society B, 1989, 51(2): 157-213.
    [8] ZHANG P G. Time series forecasting using a hybrid ARIMA and neural network model[J]. Neurocomputing, 2003, 50(1): 159-175.
    [9] SHI J, DING Z H, LEE W J, et al. Hybrid forecasting model for very-short term wind power forecasting based on grey relational analysis and wind speed distribution features[J]. IEEE Transactions on Smart Grid, 2014, 5(1): 521-526. doi: 10.1109/TSG.2013.2283269
    [10] DAHL A, BONILLA E V. Grouped Gaussian processes for solar power prediction[J]. Machine Learning, 2019, 108(8/9): 1287-1306.
    [11] TAY F, CAO L. Application of support vector machines in financial time series forecasting[J]. Omega, 2007, 29(4): 309-317.
    [12] HOSSEIN A, MOHSEN A, DAVOUD A, et al. Machine learning regression techniques for the silage maize yield prediction using time-series images of Landsat 8 OLI[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(12): 4563-4577. doi: 10.1109/JSTARS.2018.2823361
    [13] 王宁, 曹萃文. 基于XGBoost模型的炼油厂氢气网络动态多输出预测模型[J]. 华东理工大学学报(自然科学版), 2020, 46(1): 77-83.
    [14] 庞向坤, 黄越, 王振, 等. 基于相关系数的多变量异常数据段的检测[J]. 控制工程, 2020, 27(1): 194-200.
    [15] ELMAN J L. Distributed presentations, simple recurrent networks, and grammatical structure[J]. Machine Learning, 1991, 7(2): 195-225.
    [16] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735
    [17] 刘洋. 基于GRU神经网络的时间序列预测研究[D]. 成都: 成都理工大学, 2017.
    [18] JAN K, KLAUS G, FAUSTINO G. A clockwork RNN[C]//The 31st International Conference on International Conference on Machine Learning. Beijing, China: IEEE, 2014: 1863-1871.
    [19] HU J, ZHENG W. Transformation-gated LSTM: Efficient capture of short-term mutation dependencies for multivariate time series prediction tasks[C]//The International Joint Conference on Neural Networks (IJCNN). Budapest, Hungary: IEEE, 2019: 1-8.
    [20] CHEN H, WANG Y, ZHENG F, et al. Sparse modal additive model[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(6): 2373-2387.
    [21] ZHAO C, REN W, HAN M. Adaptive sparse quantization kernel least mean square algorithm for online prediction of chaotic time series[J]. Circuits, Systems, and Signal Processing, 2021, 40(1): 4346-4369.
    [22] HEWAGE P, BEHERA A, TROVATI M, et al. Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station[J]. Soft Computing, 2020, 24(1): 16453-16482.
    [23] LEWIS R, REINSEL G C. Prediction of multivariate time series by autoregressive model fitting[J]. Journal of Multivariate Analysis, 1985, 16(3): 393-411. doi: 10.1016/0047-259X(85)90027-2
    [24] 韩敏, 任伟杰, 李柏松, 等. 混沌时间序列分析与预测研究综述[J]. 信息与控制, 2020, 49(1): 28-39.
    [25] 骆科东. 短时间序列挖掘方法研究[D]. 北京: 清华大学, 2004.
    [26] 伍仕屹. 短时间序列分析及其建模方法研究[D]. 贵阳: 贵州大学, 2016.
    [27] SHIH S Y, SUN F K, LEE H Y. Temporal pattern attention for multivariate time series forecasting[J]. Machine Learning, 2019, 108(1): 1421-1441.
    [28] PADILLA W R, J GARCIA, MOLINA J M. Improving time series forecasting using information fusion in local agricultural markets[J]. Neurocomputing, 2021, 452(34): 355-373.
    [29] IAN GOODFELLOW, YOSHUA BENGIO, AARON COURVILLE. Deep Learning[M]. USA: Massachusetts Institute of Technology Press, 2016: 378-380.
    [30] SAHIN S O, KOZAT S S. Nonuniformly sampled data processingusing LSTM networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(5): 1452-1461. doi: 10.1109/TNNLS.2018.2869822
    [31] VITO S D, MASSERA E, PIGE M, et al. On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario[J]. Sensors & Actuators B Chemical, 2008, 129(2): 750-757.
    [32] LIVIERIS I E, STAVROYIANNIS S, PINTELAS E, et al. A novel validation framework to enhance deep learning models in time-series forecasting[J]. Neural Computing and Applications, 2020, 32(23): 17149-17167. doi: 10.1007/s00521-020-05169-y
    [33] NIELSEN U D, BRODTKORB A H, JENSEN J J. Response predictions using the observed autocorrelation function[J]. Marine Structures, 2018, 58(4): 31-52.
    [34] RALF O. Using neural nets in modelling vector time series[J]. Kybernetes, 1994, 23(9): 12-22. doi: 10.1108/03684929410074986
    [35] HONG W C. Chaotic particle swarm optimization algorithm in a support vector regression electric load forecasting model[J]. Energy Conversion & Management, 2009, 50(1): 105-117.
    [36] CHATZIS S P, SIAKOULIS V, PETROPOULOS A, et al. Forecasting stock market crisis events using deep and statistical machine learning techniques[J]. Expert Systems with Applications, 2018, 112(12): 353-371.
    [37] ZHOU S, MAO M, ZHOU L, et al. A shadow fault diagnosis method based on the quantitative analysis of photovoltaic output prediction error[J]. IEEE Journal of Photovoltaics, 2020, 10(4): 1158-1165. doi: 10.1109/JPHOTOV.2020.2995041
    [38] KE J, ZHENG H, YANG H, et al. Short-term forecasting of passenger demand under on-demand ride services: A spatio-temporal deep learning approach[J]. Transportation Research, 2017, 85(12): 591-608.
    [39] GENG Y, WANG X, JIANG P. Prediction of the cement grate cooler pressure in the cooling process based on a multi-model fusion neural network[J]. IEEE Access, 2020, 99(8): 115028-115040.
    [40] LI S, LIU W, LI R. The application of expert system and fuzzy control system in cement grate cooler system[C]. IEEE 7th International Conference on Software Engineering and Service Science. Beijing, China: IEEE, 2016: 770-773.
  • 加载中
图(11) / 表(2)
计量
  • 文章访问数:  45
  • HTML全文浏览量:  42
  • PDF下载量:  2
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-11-02
  • 录用日期:  2022-03-16
  • 网络出版日期:  2022-04-12

目录

    /

    返回文章
    返回